2026-02-13T06:32:07.352 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-13T06:32:07.367 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T06:32:07.448 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-13_01:43:11-rados-wip-bharath17-testing-2026-02-12-0618-tentacle-distro-default-trial/47825 branch: wip-bharath17-testing-2026-02-12-0618-tentacle description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/misc} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '47825' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: skanta-2026-02-13_01:43:11-rados-wip-bharath17-testing-2026-02-12-0618-tentacle-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath17-testing-2026-02-12-0618-tentacle ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb workunit: branch: wip-bharath17-testing-2026-02-12-0618-tentacle sha1: 3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 80 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2723 sha1: 3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath17-testing-2026-02-12-0618-tentacle suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb targets: trial040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwMabqk1KaajoCB6wauxTW5RICE2hvWciPT/ZdFfAj7hQ0dcO6CZek/jYjnXftKTP2IGmSK2Kjv63KdngXiks8= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8bec0da71becad44414c54979f64c9ef0e7099c6 timestamp: 2026-02-13_01:43:11 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3932482 2026-02-13T06:32:07.448 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb/qa; will attempt to use it 2026-02-13T06:32:07.448 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb/qa/tasks 2026-02-13T06:32:07.448 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-13T06:32:07.450 INFO:teuthology.task.internal:Checking packages... 2026-02-13T06:32:07.450 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb' 2026-02-13T06:32:07.450 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T06:32:07.450 INFO:teuthology.packaging:ref: None 2026-02-13T06:32:07.450 INFO:teuthology.packaging:tag: None 2026-02-13T06:32:07.450 INFO:teuthology.packaging:branch: wip-bharath17-testing-2026-02-12-0618-tentacle 2026-02-13T06:32:07.450 INFO:teuthology.packaging:sha1: 3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb 2026-02-13T06:32:07.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath17-testing-2026-02-12-0618-tentacle 2026-02-13T06:32:07.581 INFO:teuthology.task.internal:Found packages for ceph version 20.2.0-636.g3b40640f 2026-02-13T06:32:07.582 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-13T06:32:07.594 INFO:teuthology.task.internal:no buildpackages task found 2026-02-13T06:32:07.595 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-13T06:32:07.609 INFO:teuthology.task.internal:Saving configuration 2026-02-13T06:32:07.616 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-13T06:32:07.628 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-13T06:32:07.709 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial040.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-13_01:43:11-rados-wip-bharath17-testing-2026-02-12-0618-tentacle-distro-default-trial/47825', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-13 06:28:21.725507', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwMabqk1KaajoCB6wauxTW5RICE2hvWciPT/ZdFfAj7hQ0dcO6CZek/jYjnXftKTP2IGmSK2Kjv63KdngXiks8='} 2026-02-13T06:32:07.709 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-13T06:32:07.721 INFO:teuthology.task.internal:roles: ubuntu@trial040.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-13T06:32:07.721 INFO:teuthology.run_tasks:Running task console_log... 2026-02-13T06:32:07.808 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x73a9db3b1bc0>, signals=[15]) 2026-02-13T06:32:07.808 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-13T06:32:07.820 INFO:teuthology.task.internal:Opening connections... 2026-02-13T06:32:07.820 DEBUG:teuthology.task.internal:connecting to ubuntu@trial040.front.sepia.ceph.com 2026-02-13T06:32:07.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T06:32:07.883 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-13T06:32:07.899 DEBUG:teuthology.orchestra.run.trial040:> uname -m 2026-02-13T06:32:07.957 INFO:teuthology.orchestra.run.trial040.stdout:x86_64 2026-02-13T06:32:07.957 DEBUG:teuthology.orchestra.run.trial040:> cat /etc/os-release 2026-02-13T06:32:08.014 INFO:teuthology.orchestra.run.trial040.stdout:NAME="CentOS Stream" 2026-02-13T06:32:08.014 INFO:teuthology.orchestra.run.trial040.stdout:VERSION="9" 2026-02-13T06:32:08.014 INFO:teuthology.orchestra.run.trial040.stdout:ID="centos" 2026-02-13T06:32:08.014 INFO:teuthology.orchestra.run.trial040.stdout:ID_LIKE="rhel fedora" 2026-02-13T06:32:08.014 INFO:teuthology.orchestra.run.trial040.stdout:VERSION_ID="9" 2026-02-13T06:32:08.014 INFO:teuthology.orchestra.run.trial040.stdout:PLATFORM_ID="platform:el9" 2026-02-13T06:32:08.014 INFO:teuthology.orchestra.run.trial040.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-13T06:32:08.015 INFO:teuthology.orchestra.run.trial040.stdout:ANSI_COLOR="0;31" 2026-02-13T06:32:08.015 INFO:teuthology.orchestra.run.trial040.stdout:LOGO="fedora-logo-icon" 2026-02-13T06:32:08.015 INFO:teuthology.orchestra.run.trial040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-13T06:32:08.015 INFO:teuthology.orchestra.run.trial040.stdout:HOME_URL="https://centos.org/" 2026-02-13T06:32:08.015 INFO:teuthology.orchestra.run.trial040.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-13T06:32:08.015 INFO:teuthology.orchestra.run.trial040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-13T06:32:08.015 INFO:teuthology.orchestra.run.trial040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-13T06:32:08.016 INFO:teuthology.lock.ops:Updating trial040.front.sepia.ceph.com on lock server 2026-02-13T06:32:08.114 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-13T06:32:08.126 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-13T06:32:08.138 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-13T06:32:08.138 DEBUG:teuthology.orchestra.run.trial040:> test '!' -e /home/ubuntu/cephtest 2026-02-13T06:32:08.153 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-13T06:32:08.165 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-13T06:32:08.165 DEBUG:teuthology.orchestra.run.trial040:> test -z $(ls -A /var/lib/ceph) 2026-02-13T06:32:08.210 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-13T06:32:08.315 INFO:teuthology.run_tasks:Running task kernel... 2026-02-13T06:32:08.337 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-13T06:32:08.337 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-13T06:32:08.337 DEBUG:teuthology.orchestra.run.trial040:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-13T06:32:08.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T06:32:08.352 DEBUG:teuthology.orchestra.run.trial040:> uname -r 2026-02-13T06:32:08.408 INFO:teuthology.orchestra.run.trial040.stdout:5.14.0-645.el9.x86_64 2026-02-13T06:32:08.408 INFO:teuthology.task.kernel:Running kernel on trial040: 5.14.0-645.el9.x86_64 2026-02-13T06:32:08.408 DEBUG:teuthology.orchestra.run.trial040:> sudo yum install -y kernel 2026-02-13T06:32:09.583 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-13T06:32:13.300 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 27 MB 00:03 2026-02-13T06:32:17.086 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 7.8 MB 00:01 2026-02-13T06:32:17.940 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - Extras packages 147 kB/s | 20 kB 00:00 2026-02-13T06:32:18.575 INFO:teuthology.orchestra.run.trial040.stdout:Extra Packages for Enterprise Linux 36 MB/s | 20 MB 00:00 2026-02-13T06:32:21.813 INFO:teuthology.orchestra.run.trial040.stdout:lab-extras 1.6 MB/s | 50 kB 00:00 2026-02-13T06:32:22.871 INFO:teuthology.orchestra.run.trial040.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-13T06:32:22.890 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:32:22.891 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout: Package Architecture Version Repository Size 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout:Installing dependencies: 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:32:22.892 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:32:22.893 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:32:22.893 INFO:teuthology.orchestra.run.trial040.stdout:Install 4 Packages 2026-02-13T06:32:22.893 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:32:22.893 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 88 M 2026-02-13T06:32:22.893 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 135 M 2026-02-13T06:32:22.893 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:32:23.827 INFO:teuthology.orchestra.run.trial040.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 906 kB/s | 768 kB 00:00 2026-02-13T06:32:24.290 INFO:teuthology.orchestra.run.trial040.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 13 MB/s | 17 MB 00:01 2026-02-13T06:32:24.782 INFO:teuthology.orchestra.run.trial040.stdout:(3/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 22 MB/s | 40 MB 00:01 2026-02-13T06:32:25.427 INFO:teuthology.orchestra.run.trial040.stdout:(4/4): kernel-modules-core-5.14.0-677.el9.x86_6 19 MB/s | 31 MB 00:01 2026-02-13T06:32:25.428 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:32:25.428 INFO:teuthology.orchestra.run.trial040.stdout:Total 35 MB/s | 88 MB 00:02 2026-02-13T06:32:25.617 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:32:25.819 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:32:25.819 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:32:26.165 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:32:26.165 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:32:27.254 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:32:27.539 INFO:teuthology.orchestra.run.trial040.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-13T06:32:27.559 INFO:teuthology.orchestra.run.trial040.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T06:32:29.327 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T06:32:29.356 INFO:teuthology.orchestra.run.trial040.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T06:32:32.379 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T06:32:32.386 INFO:teuthology.orchestra.run.trial040.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T06:32:35.627 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T06:33:06.374 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T06:33:06.397 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-13T06:33:06.461 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T06:33:06.461 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-13T06:33:06.462 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T06:33:06.462 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T06:33:06.513 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T06:33:06.513 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:33:06.514 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:33:06.514 INFO:teuthology.orchestra.run.trial040.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-13T06:33:06.514 INFO:teuthology.orchestra.run.trial040.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-13T06:33:06.514 INFO:teuthology.orchestra.run.trial040.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-13T06:33:06.514 INFO:teuthology.orchestra.run.trial040.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-13T06:33:06.514 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:33:06.514 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:33:06.573 DEBUG:teuthology.orchestra.run.trial040:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T06:33:06.599 INFO:teuthology.orchestra.run.trial040.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T06:33:06.600 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T06:33:06.600 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-13T06:33:06.600 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-13T06:33:06.600 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-13T06:33:06.600 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-13T06:33:06.601 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial040.front.sepia.ceph.com, path=None, version=distro) 2026-02-13T06:33:06.601 DEBUG:teuthology.orchestra.run.trial040:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T06:33:06.663 INFO:teuthology.orchestra.run.trial040.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T06:33:06.664 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T06:33:06.664 DEBUG:teuthology.orchestra.run.trial040:> sudo rpm -qi grub2-tools 2026-02-13T06:33:06.733 INFO:teuthology.orchestra.run.trial040.stdout:Name : grub2-tools 2026-02-13T06:33:06.733 INFO:teuthology.orchestra.run.trial040.stdout:Epoch : 1 2026-02-13T06:33:06.733 INFO:teuthology.orchestra.run.trial040.stdout:Version : 2.06 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Release : 118.el9 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Architecture: x86_64 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Group : Unspecified 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Size : 8393909 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:License : GPLv3+ 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Packager : builder@centos.org 2026-02-13T06:33:06.734 INFO:teuthology.orchestra.run.trial040.stdout:Vendor : CentOS 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout:Summary : Support tools for GRUB. 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout:Description : 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout:hardware devices. 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:33:06.735 INFO:teuthology.orchestra.run.trial040.stdout:This subpackage provides tools for support of all platforms. 2026-02-13T06:33:06.736 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-13T06:33:06.737 INFO:teuthology.task.kernel:Updating grub on trial040 to boot 5.14.0-677.el9.x86_64 2026-02-13T06:33:06.737 DEBUG:teuthology.orchestra.run.trial040:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-13T06:33:06.800 INFO:teuthology.orchestra.run.trial040.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-13T06:33:06.801 DEBUG:teuthology.orchestra.run.trial040:> sudo command -v grubby && echo yes || echo no 2026-02-13T06:33:06.866 INFO:teuthology.orchestra.run.trial040.stdout:/sbin/grubby 2026-02-13T06:33:06.867 INFO:teuthology.orchestra.run.trial040.stdout:yes 2026-02-13T06:33:06.868 DEBUG:teuthology.orchestra.run.trial040:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-13T06:33:06.931 INFO:teuthology.orchestra.run.trial040.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-13T06:33:06.932 DEBUG:teuthology.orchestra.run.trial040:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-13T06:33:07.006 DEBUG:teuthology.orchestra.run.trial040:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-13T06:33:07.213 INFO:teuthology.orchestra.run.trial040.stderr:Generating grub configuration file ... 2026-02-13T06:33:07.275 INFO:teuthology.orchestra.run.trial040.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-13T06:33:07.281 INFO:teuthology.orchestra.run.trial040.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-13T06:33:07.530 INFO:teuthology.orchestra.run.trial040.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-13T06:33:07.536 INFO:teuthology.orchestra.run.trial040.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-13T06:33:07.567 INFO:teuthology.orchestra.run.trial040.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-13T06:33:07.574 INFO:teuthology.orchestra.run.trial040.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-13T06:33:07.597 INFO:teuthology.orchestra.run.trial040.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-13T06:33:07.603 INFO:teuthology.orchestra.run.trial040.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-13T06:33:07.684 INFO:teuthology.orchestra.run.trial040.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-13T06:33:07.692 INFO:teuthology.orchestra.run.trial040.stderr:done 2026-02-13T06:33:07.695 DEBUG:teuthology.orchestra.run.trial040:> test -d /sys/firmware/efi 2026-02-13T06:33:07.710 DEBUG:teuthology.orchestra.run.trial040:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-13T06:33:07.775 INFO:teuthology.orchestra.run.trial040.stdout:/boot/efi/EFI/BOOT 2026-02-13T06:33:07.775 INFO:teuthology.orchestra.run.trial040.stdout:/boot/efi/EFI/centos 2026-02-13T06:33:07.777 DEBUG:teuthology.orchestra.run.trial040:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-13T06:33:07.843 INFO:teuthology.orchestra.run.trial040.stdout:no 2026-02-13T06:33:07.843 DEBUG:teuthology.orchestra.run.trial040:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-13T06:33:07.907 INFO:teuthology.orchestra.run.trial040.stdout:yes 2026-02-13T06:33:07.908 DEBUG:teuthology.orchestra.run.trial040:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-13T06:33:07.971 DEBUG:teuthology.orchestra.run.trial040:> sudo shutdown -r now 2026-02-13T06:33:08.044 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-13T06:33:38.024 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-13T06:33:38.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial040.front.sepia.ceph.com' 2026-02-13T06:33:38.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T06:34:38.086 DEBUG:teuthology.orchestra.remote:timed out 2026-02-13T06:34:47.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial040.front.sepia.ceph.com' 2026-02-13T06:34:47.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T06:34:47.227 DEBUG:teuthology.orchestra.run.trial040:> true 2026-02-13T06:34:47.390 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial040.front.sepia.ceph.com' 2026-02-13T06:34:47.390 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-13T06:34:47.390 DEBUG:teuthology.orchestra.run.trial040:> uname -r 2026-02-13T06:34:47.446 INFO:teuthology.orchestra.run.trial040.stdout:5.14.0-677.el9.x86_64 2026-02-13T06:34:47.447 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-13T06:34:47.447 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-13T06:34:47.447 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-13T06:34:48.448 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-13T06:34:48.448 DEBUG:teuthology.orchestra.run.trial040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-13T06:34:48.477 INFO:teuthology.orchestra.run.trial040.stdout:ttyS1 2026-02-13T06:34:48.508 DEBUG:teuthology.parallel:result is None 2026-02-13T06:34:48.508 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-13T06:34:48.521 INFO:teuthology.task.internal:Creating test directory... 2026-02-13T06:34:48.521 DEBUG:teuthology.orchestra.run.trial040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-13T06:34:48.565 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-13T06:34:48.576 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-13T06:34:48.589 INFO:teuthology.task.internal:Creating archive directory... 2026-02-13T06:34:48.589 DEBUG:teuthology.orchestra.run.trial040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-13T06:34:48.627 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-13T06:34:48.639 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-13T06:34:48.639 DEBUG:teuthology.orchestra.run.trial040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-13T06:34:48.685 INFO:teuthology.orchestra.run.trial040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:34:48.693 INFO:teuthology.orchestra.run.trial040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:34:48.695 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-13T06:34:48.707 INFO:teuthology.task.internal:Configuring sudo... 2026-02-13T06:34:48.707 DEBUG:teuthology.orchestra.run.trial040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-13T06:34:48.761 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-13T06:34:48.778 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-13T06:34:48.778 DEBUG:teuthology.orchestra.run.trial040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-13T06:34:48.819 DEBUG:teuthology.orchestra.run.trial040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T06:34:49.003 DEBUG:teuthology.orchestra.run.trial040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T06:34:49.027 DEBUG:teuthology.orchestra.run.trial040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T06:34:49.143 DEBUG:teuthology.orchestra.run.trial040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T06:34:49.166 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-13T06:34:49.166 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-13T06:34:49.230 DEBUG:teuthology.orchestra.run.trial040:> sudo service rsyslog restart 2026-02-13T06:34:49.298 INFO:teuthology.orchestra.run.trial040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T06:34:49.821 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-13T06:34:49.834 INFO:teuthology.task.internal:Starting timer... 2026-02-13T06:34:49.834 INFO:teuthology.run_tasks:Running task pcp... 2026-02-13T06:34:49.857 INFO:teuthology.run_tasks:Running task selinux... 2026-02-13T06:34:49.874 DEBUG:teuthology.orchestra.run.trial040:> sudo service auditd rotate 2026-02-13T06:34:49.904 INFO:teuthology.orchestra.run.trial040.stdout:Rotating logs: 2026-02-13T06:34:49.906 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-13T06:34:49.906 DEBUG:teuthology.orchestra.run.trial040:> /usr/sbin/getenforce 2026-02-13T06:34:49.963 INFO:teuthology.orchestra.run.trial040.stdout:Permissive 2026-02-13T06:34:49.963 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial040.front.sepia.ceph.com': 'permissive'} 2026-02-13T06:34:49.963 DEBUG:teuthology.orchestra.run.trial040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-13T06:34:50.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T06:34:50.028 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-13T06:34:50.028 DEBUG:teuthology.orchestra.run.trial040:> sudo /usr/sbin/setenforce permissive 2026-02-13T06:34:50.091 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-13T06:34:50.107 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-13T06:34:50.110 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-13T06:34:50.226 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-13T06:34:50.236 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-13T06:34:50.237 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial040.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-13T06:36:00.198 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial040.front.sepia.ceph.com')] 2026-02-13T06:36:00.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial040.front.sepia.ceph.com' 2026-02-13T06:36:00.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T06:36:00.308 DEBUG:teuthology.orchestra.run.trial040:> true 2026-02-13T06:36:00.385 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial040.front.sepia.ceph.com' 2026-02-13T06:36:00.385 INFO:teuthology.run_tasks:Running task clock... 2026-02-13T06:36:00.399 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-13T06:36:00.399 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-13T06:36:00.399 DEBUG:teuthology.orchestra.run.trial040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T06:36:00.451 INFO:teuthology.orchestra.run.trial040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-13T06:36:00.463 INFO:teuthology.orchestra.run.trial040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-13T06:36:00.483 INFO:teuthology.orchestra.run.trial040.stderr:sudo: ntpd: command not found 2026-02-13T06:36:00.492 INFO:teuthology.orchestra.run.trial040.stdout:506 Cannot talk to daemon 2026-02-13T06:36:00.503 INFO:teuthology.orchestra.run.trial040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-13T06:36:00.514 INFO:teuthology.orchestra.run.trial040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-13T06:36:00.562 INFO:teuthology.orchestra.run.trial040.stderr:bash: line 1: ntpq: command not found 2026-02-13T06:36:00.564 INFO:teuthology.orchestra.run.trial040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T06:36:00.564 INFO:teuthology.orchestra.run.trial040.stdout:=============================================================================== 2026-02-13T06:36:00.565 INFO:teuthology.run_tasks:Running task install... 2026-02-13T06:36:00.579 DEBUG:teuthology.task.install:project ceph 2026-02-13T06:36:00.579 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb'}} 2026-02-13T06:36:00.579 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb'} 2026-02-13T06:36:00.579 INFO:teuthology.task.install:Using flavor: default 2026-02-13T06:36:00.588 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-13T06:36:00.588 INFO:teuthology.task.install:extra packages: [] 2026-02-13T06:36:00.588 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb', 'tag': None, 'wait_for_package': False} 2026-02-13T06:36:00.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb 2026-02-13T06:36:00.760 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath17-testing-2026-02-12-0618-tentacle/3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb/centos/9/flavors/default/ 2026-02-13T06:36:00.760 INFO:teuthology.task.install.rpm:Package version is 20.2.0-636.g3b40640f 2026-02-13T06:36:01.066 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath17-testing-2026-02-12-0618-tentacle/3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath17-testing-2026-02-12-0618-tentacle/3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath17-testing-2026-02-12-0618-tentacle/3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-13T06:36:01.066 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-13T06:36:01.066 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-13T06:36:01.092 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-13T06:36:01.092 DEBUG:teuthology.orchestra.run.trial040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-13T06:36:01.157 DEBUG:teuthology.orchestra.run.trial040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-13T06:36:01.230 DEBUG:teuthology.orchestra.run.trial040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-13T06:36:01.295 INFO:teuthology.orchestra.run.trial040.stdout:check_obsoletes = 1 2026-02-13T06:36:01.296 DEBUG:teuthology.orchestra.run.trial040:> sudo yum clean all 2026-02-13T06:36:01.445 INFO:teuthology.orchestra.run.trial040.stdout:41 files removed 2026-02-13T06:36:01.459 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-radosgw 2026-02-13T06:36:01.744 INFO:teuthology.orchestra.run.trial040.stdout:ceph packages for x86_64 626 kB/s | 89 kB 00:00 2026-02-13T06:36:01.883 INFO:teuthology.orchestra.run.trial040.stdout:ceph noarch packages 164 kB/s | 19 kB 00:00 2026-02-13T06:36:02.011 INFO:teuthology.orchestra.run.trial040.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-13T06:36:02.867 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-13T06:36:06.803 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 27 MB 00:03 2026-02-13T06:36:10.399 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 7.8 MB 00:01 2026-02-13T06:36:11.310 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - Extras packages 110 kB/s | 20 kB 00:00 2026-02-13T06:36:12.023 INFO:teuthology.orchestra.run.trial040.stdout:Extra Packages for Enterprise Linux 32 MB/s | 20 MB 00:00 2026-02-13T06:36:15.269 INFO:teuthology.orchestra.run.trial040.stdout:lab-extras 1.3 MB/s | 50 kB 00:00 2026-02-13T06:36:16.557 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:36:16.559 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:36:16.559 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repository Size 2026-02-13T06:36:16.559 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:36:16.559 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:36:16.559 INFO:teuthology.orchestra.run.trial040.stdout: ceph-radosgw x86_64 2:20.2.0-636.g3b40640f.el9 ceph 24 M 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout:Upgrading: 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: librados2 x86_64 2:20.2.0-636.g3b40640f.el9 ceph 3.5 M 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: librbd1 x86_64 2:20.2.0-636.g3b40640f.el9 ceph 2.8 M 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout:Installing dependencies: 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: ceph-base x86_64 2:20.2.0-636.g3b40640f.el9 ceph 5.9 M 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: ceph-common x86_64 2:20.2.0-636.g3b40640f.el9 ceph 24 M 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: ceph-selinux x86_64 2:20.2.0-636.g3b40640f.el9 ceph 25 k 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-13T06:36:16.560 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs2 x86_64 2:20.2.0-636.g3b40640f.el9 ceph 870 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: libradosstriper1 x86_64 2:20.2.0-636.g3b40640f.el9 ceph 249 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: librgw2 x86_64 2:20.2.0-636.g3b40640f.el9 ceph 6.4 M 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-13T06:36:16.561 INFO:teuthology.orchestra.run.trial040.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: python3-ceph-argparse x86_64 2:20.2.0-636.g3b40640f.el9 ceph 45 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: python3-ceph-common x86_64 2:20.2.0-636.g3b40640f.el9 ceph 175 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: python3-cephfs x86_64 2:20.2.0-636.g3b40640f.el9 ceph 163 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: python3-rados x86_64 2:20.2.0-636.g3b40640f.el9 ceph 324 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: python3-rbd x86_64 2:20.2.0-636.g3b40640f.el9 ceph 306 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: python3-rgw x86_64 2:20.2.0-636.g3b40640f.el9 ceph 99 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout:Installing weak dependencies: 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs-daemon x86_64 2:20.2.0-636.g3b40640f.el9 ceph 35 k 2026-02-13T06:36:16.562 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs-proxy2 x86_64 2:20.2.0-636.g3b40640f.el9 ceph 24 k 2026-02-13T06:36:16.563 INFO:teuthology.orchestra.run.trial040.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-13T06:36:16.563 INFO:teuthology.orchestra.run.trial040.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-13T06:36:16.563 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:16.563 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:36:16.563 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:36:16.563 INFO:teuthology.orchestra.run.trial040.stdout:Install 35 Packages 2026-02-13T06:36:16.563 INFO:teuthology.orchestra.run.trial040.stdout:Upgrade 2 Packages 2026-02-13T06:36:16.563 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:16.564 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 79 M 2026-02-13T06:36:16.564 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:36:17.525 INFO:teuthology.orchestra.run.trial040.stdout:(1/37): ceph-radosgw-20.2.0-636.g3b40640f.el9.x 34 MB/s | 24 MB 00:00 2026-02-13T06:36:17.537 INFO:teuthology.orchestra.run.trial040.stdout:(2/37): ceph-selinux-20.2.0-636.g3b40640f.el9.x 2.1 MB/s | 25 kB 00:00 2026-02-13T06:36:17.577 INFO:teuthology.orchestra.run.trial040.stdout:(3/37): libcephfs-daemon-20.2.0-636.g3b40640f.e 876 kB/s | 35 kB 00:00 2026-02-13T06:36:17.626 INFO:teuthology.orchestra.run.trial040.stdout:(4/37): libcephfs-proxy2-20.2.0-636.g3b40640f.e 495 kB/s | 24 kB 00:00 2026-02-13T06:36:17.695 INFO:teuthology.orchestra.run.trial040.stdout:(5/37): libcephfs2-20.2.0-636.g3b40640f.el9.x86 12 MB/s | 870 kB 00:00 2026-02-13T06:36:17.754 INFO:teuthology.orchestra.run.trial040.stdout:(6/37): libradosstriper1-20.2.0-636.g3b40640f.e 4.1 MB/s | 249 kB 00:00 2026-02-13T06:36:17.940 INFO:teuthology.orchestra.run.trial040.stdout:(7/37): ceph-base-20.2.0-636.g3b40640f.el9.x86_ 5.3 MB/s | 5.9 MB 00:01 2026-02-13T06:36:18.001 INFO:teuthology.orchestra.run.trial040.stdout:(8/37): python3-ceph-argparse-20.2.0-636.g3b406 735 kB/s | 45 kB 00:00 2026-02-13T06:36:18.055 INFO:teuthology.orchestra.run.trial040.stdout:(9/37): python3-ceph-common-20.2.0-636.g3b40640 3.2 MB/s | 175 kB 00:00 2026-02-13T06:36:18.118 INFO:teuthology.orchestra.run.trial040.stdout:(10/37): python3-cephfs-20.2.0-636.g3b40640f.el 2.5 MB/s | 163 kB 00:00 2026-02-13T06:36:18.175 INFO:teuthology.orchestra.run.trial040.stdout:(11/37): python3-rados-20.2.0-636.g3b40640f.el9 5.5 MB/s | 324 kB 00:00 2026-02-13T06:36:18.231 INFO:teuthology.orchestra.run.trial040.stdout:(12/37): python3-rbd-20.2.0-636.g3b40640f.el9.x 5.4 MB/s | 306 kB 00:00 2026-02-13T06:36:18.273 INFO:teuthology.orchestra.run.trial040.stdout:(13/37): python3-rgw-20.2.0-636.g3b40640f.el9.x 2.3 MB/s | 99 kB 00:00 2026-02-13T06:36:18.337 INFO:teuthology.orchestra.run.trial040.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 518 kB/s | 33 kB 00:00 2026-02-13T06:36:18.462 INFO:teuthology.orchestra.run.trial040.stdout:(15/37): librgw2-20.2.0-636.g3b40640f.el9.x86_6 9.0 MB/s | 6.4 MB 00:00 2026-02-13T06:36:18.502 INFO:teuthology.orchestra.run.trial040.stdout:(16/37): nvme-cli-2.13-1.el9.x86_64.rpm 6.6 MB/s | 1.1 MB 00:00 2026-02-13T06:36:18.560 INFO:teuthology.orchestra.run.trial040.stdout:(17/37): ceph-common-20.2.0-636.g3b40640f.el9.x 14 MB/s | 24 MB 00:01 2026-02-13T06:36:18.564 INFO:teuthology.orchestra.run.trial040.stdout:(18/37): boost-program-options-1.75.0-13.el9.x8 1.7 MB/s | 104 kB 00:00 2026-02-13T06:36:18.580 INFO:teuthology.orchestra.run.trial040.stdout:(19/37): pciutils-3.7.0-7.el9.x86_64.rpm 787 kB/s | 93 kB 00:00 2026-02-13T06:36:18.586 INFO:teuthology.orchestra.run.trial040.stdout:(20/37): libnbd-1.20.3-4.el9.x86_64.rpm 6.2 MB/s | 164 kB 00:00 2026-02-13T06:36:18.590 INFO:teuthology.orchestra.run.trial040.stdout:(21/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.0 MB/s | 160 kB 00:00 2026-02-13T06:36:18.603 INFO:teuthology.orchestra.run.trial040.stdout:(22/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.0 MB/s | 45 kB 00:00 2026-02-13T06:36:18.614 INFO:teuthology.orchestra.run.trial040.stdout:(23/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 12 MB/s | 292 kB 00:00 2026-02-13T06:36:18.619 INFO:teuthology.orchestra.run.trial040.stdout:(24/37): librdkafka-1.6.1-102.el9.x86_64.rpm 20 MB/s | 662 kB 00:00 2026-02-13T06:36:18.629 INFO:teuthology.orchestra.run.trial040.stdout:(25/37): qatlib-25.08.0-2.el9.x86_64.rpm 8.8 MB/s | 240 kB 00:00 2026-02-13T06:36:18.638 INFO:teuthology.orchestra.run.trial040.stdout:(26/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.5 MB/s | 37 kB 00:00 2026-02-13T06:36:18.640 INFO:teuthology.orchestra.run.trial040.stdout:(27/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 3.0 MB/s | 66 kB 00:00 2026-02-13T06:36:18.812 INFO:teuthology.orchestra.run.trial040.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 144 kB/s | 25 kB 00:00 2026-02-13T06:36:18.921 INFO:teuthology.orchestra.run.trial040.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2026-02-13T06:36:18.991 INFO:teuthology.orchestra.run.trial040.stdout:(30/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 851 kB/s | 308 kB 00:00 2026-02-13T06:36:19.001 INFO:teuthology.orchestra.run.trial040.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 840 kB/s | 67 kB 00:00 2026-02-13T06:36:19.129 INFO:teuthology.orchestra.run.trial040.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2026-02-13T06:36:19.146 INFO:teuthology.orchestra.run.trial040.stdout:(33/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 5.3 MB/s | 838 kB 00:00 2026-02-13T06:36:19.201 INFO:teuthology.orchestra.run.trial040.stdout:(34/37): libarrow-9.0.0-15.el9.x86_64.rpm 7.8 MB/s | 4.4 MB 00:00 2026-02-13T06:36:19.287 INFO:teuthology.orchestra.run.trial040.stdout:(35/37): librbd1-20.2.0-636.g3b40640f.el9.x86_6 33 MB/s | 2.8 MB 00:00 2026-02-13T06:36:19.324 INFO:teuthology.orchestra.run.trial040.stdout:(36/37): librados2-20.2.0-636.g3b40640f.el9.x86 20 MB/s | 3.5 MB 00:00 2026-02-13T06:36:19.342 INFO:teuthology.orchestra.run.trial040.stdout:(37/37): thrift-0.15.0-4.el9.x86_64.rpm 7.4 MB/s | 1.6 MB 00:00 2026-02-13T06:36:19.344 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:36:19.345 INFO:teuthology.orchestra.run.trial040.stdout:Total 28 MB/s | 79 MB 00:02 2026-02-13T06:36:19.447 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:36:19.465 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:36:19.466 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:36:19.762 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:36:19.762 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:36:20.104 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:36:20.116 INFO:teuthology.orchestra.run.trial040.stdout: Installing : thrift-0.15.0-4.el9.x86_64 1/39 2026-02-13T06:36:20.272 INFO:teuthology.orchestra.run.trial040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 2/39 2026-02-13T06:36:20.276 INFO:teuthology.orchestra.run.trial040.stdout: Upgrading : librados2-2:20.2.0-636.g3b40640f.el9.x86_64 3/39 2026-02-13T06:36:20.326 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librados2-2:20.2.0-636.g3b40640f.el9.x86_64 3/39 2026-02-13T06:36:20.328 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libcephfs2-2:20.2.0-636.g3b40640f.el9.x86_64 4/39 2026-02-13T06:36:20.340 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libcephfs2-2:20.2.0-636.g3b40640f.el9.x86_64 4/39 2026-02-13T06:36:20.343 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libcephfs-daemon-2:20.2.0-636.g3b40640f.el9.x86_64 5/39 2026-02-13T06:36:20.344 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libcephfs-proxy2-2:20.2.0-636.g3b40640f.el9.x86_64 6/39 2026-02-13T06:36:20.370 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libcephfs-proxy2-2:20.2.0-636.g3b40640f.el9.x86_64 6/39 2026-02-13T06:36:20.372 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rados-2:20.2.0-636.g3b40640f.el9.x86_64 7/39 2026-02-13T06:36:20.380 INFO:teuthology.orchestra.run.trial040.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/39 2026-02-13T06:36:20.382 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/39 2026-02-13T06:36:20.385 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 10/39 2026-02-13T06:36:20.389 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 11/39 2026-02-13T06:36:20.398 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-ceph-argparse-2:20.2.0-636.g3b40640f.el9.x 12/39 2026-02-13T06:36:20.410 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cephfs-2:20.2.0-636.g3b40640f.el9.x86_64 13/39 2026-02-13T06:36:20.411 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libradosstriper1-2:20.2.0-636.g3b40640f.el9.x86_64 14/39 2026-02-13T06:36:20.425 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libradosstriper1-2:20.2.0-636.g3b40640f.el9.x86_64 14/39 2026-02-13T06:36:20.458 INFO:teuthology.orchestra.run.trial040.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-13T06:36:20.462 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-13T06:36:20.468 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-13T06:36:20.470 INFO:teuthology.orchestra.run.trial040.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-13T06:36:20.475 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-13T06:36:20.600 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-13T06:36:20.601 INFO:teuthology.orchestra.run.trial040.stdout: Upgrading : librbd1-2:20.2.0-636.g3b40640f.el9.x86_64 21/39 2026-02-13T06:36:20.630 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librbd1-2:20.2.0-636.g3b40640f.el9.x86_64 21/39 2026-02-13T06:36:20.632 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rbd-2:20.2.0-636.g3b40640f.el9.x86_64 22/39 2026-02-13T06:36:20.639 INFO:teuthology.orchestra.run.trial040.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-13T06:36:20.927 INFO:teuthology.orchestra.run.trial040.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-13T06:36:20.928 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librgw2-2:20.2.0-636.g3b40640f.el9.x86_64 25/39 2026-02-13T06:36:20.945 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librgw2-2:20.2.0-636.g3b40640f.el9.x86_64 25/39 2026-02-13T06:36:20.949 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rgw-2:20.2.0-636.g3b40640f.el9.x86_64 26/39 2026-02-13T06:36:20.953 INFO:teuthology.orchestra.run.trial040.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-13T06:36:20.955 INFO:teuthology.orchestra.run.trial040.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-13T06:36:20.956 INFO:teuthology.orchestra.run.trial040.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-13T06:36:20.973 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-13T06:36:21.085 INFO:teuthology.orchestra.run.trial040.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-13T06:36:21.087 INFO:teuthology.orchestra.run.trial040.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-13T06:36:21.350 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-13T06:36:21.350 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-13T06:36:21.350 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:21.393 INFO:teuthology.orchestra.run.trial040.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-13T06:36:21.395 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-ceph-common-2:20.2.0-636.g3b40640f.el9.x86 33/39 2026-02-13T06:36:22.490 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-common-2:20.2.0-636.g3b40640f.el9.x86_64 34/39 2026-02-13T06:36:22.492 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-common-2:20.2.0-636.g3b40640f.el9.x86_64 34/39 2026-02-13T06:36:22.791 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-common-2:20.2.0-636.g3b40640f.el9.x86_64 34/39 2026-02-13T06:36:22.795 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-base-2:20.2.0-636.g3b40640f.el9.x86_64 35/39 2026-02-13T06:36:22.833 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-base-2:20.2.0-636.g3b40640f.el9.x86_64 35/39 2026-02-13T06:36:22.833 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-13T06:36:22.833 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-13T06:36:22.833 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:22.837 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-selinux-2:20.2.0-636.g3b40640f.el9.x86_64 36/39 2026-02-13T06:36:27.382 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-selinux-2:20.2.0-636.g3b40640f.el9.x86_64 36/39 2026-02-13T06:36:27.382 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /sys 2026-02-13T06:36:27.382 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /proc 2026-02-13T06:36:27.382 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /mnt 2026-02-13T06:36:27.382 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /var/tmp 2026-02-13T06:36:27.382 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /home 2026-02-13T06:36:27.382 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /root 2026-02-13T06:36:27.382 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /tmp 2026-02-13T06:36:27.382 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:28.423 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-radosgw-2:20.2.0-636.g3b40640f.el9.x86_64 37/39 2026-02-13T06:36:28.447 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-radosgw-2:20.2.0-636.g3b40640f.el9.x86_64 37/39 2026-02-13T06:36:28.447 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T06:36:28.447 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-13T06:36:28.448 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T06:36:28.448 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T06:36:28.448 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:28.449 INFO:teuthology.orchestra.run.trial040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-13T06:36:28.462 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-13T06:36:28.462 INFO:teuthology.orchestra.run.trial040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-13T06:36:28.517 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-13T06:36:28.517 INFO:teuthology.orchestra.run.trial040.stdout:Creating group 'qat' with GID 992. 2026-02-13T06:36:28.517 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:28.823 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-base-2:20.2.0-636.g3b40640f.el9.x86_64 1/39 2026-02-13T06:36:28.823 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-common-2:20.2.0-636.g3b40640f.el9.x86_64 2/39 2026-02-13T06:36:28.823 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-radosgw-2:20.2.0-636.g3b40640f.el9.x86_64 3/39 2026-02-13T06:36:28.823 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-selinux-2:20.2.0-636.g3b40640f.el9.x86_64 4/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libcephfs-daemon-2:20.2.0-636.g3b40640f.el9.x86_64 5/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libcephfs-proxy2-2:20.2.0-636.g3b40640f.el9.x86_64 6/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libcephfs2-2:20.2.0-636.g3b40640f.el9.x86_64 7/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libradosstriper1-2:20.2.0-636.g3b40640f.el9.x86_64 8/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librgw2-2:20.2.0-636.g3b40640f.el9.x86_64 9/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-ceph-argparse-2:20.2.0-636.g3b40640f.el9.x 10/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-ceph-common-2:20.2.0-636.g3b40640f.el9.x86 11/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cephfs-2:20.2.0-636.g3b40640f.el9.x86_64 12/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rados-2:20.2.0-636.g3b40640f.el9.x86_64 13/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rbd-2:20.2.0-636.g3b40640f.el9.x86_64 14/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rgw-2:20.2.0-636.g3b40640f.el9.x86_64 15/39 2026-02-13T06:36:28.824 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 17/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-13T06:36:28.825 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-13T06:36:28.826 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-13T06:36:28.826 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-13T06:36:28.826 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-13T06:36:28.826 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-13T06:36:28.826 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-13T06:36:28.826 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-13T06:36:28.826 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librados2-2:20.2.0-636.g3b40640f.el9.x86_64 36/39 2026-02-13T06:36:28.826 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-13T06:36:28.826 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librbd1-2:20.2.0-636.g3b40640f.el9.x86_64 38/39 2026-02-13T06:36:28.885 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout:Upgraded: 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout: librados2-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout: librbd1-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout: ceph-base-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout: ceph-common-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout: ceph-radosgw-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.886 INFO:teuthology.orchestra.run.trial040.stdout: ceph-selinux-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs-daemon-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs-proxy2-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs2-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: libradosstriper1-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.887 INFO:teuthology.orchestra.run.trial040.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: librgw2-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: python3-ceph-argparse-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: python3-ceph-common-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: python3-cephfs-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: python3-rados-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: python3-rbd-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: python3-rgw-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:28.888 INFO:teuthology.orchestra.run.trial040.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-13T06:36:28.889 INFO:teuthology.orchestra.run.trial040.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-13T06:36:28.889 INFO:teuthology.orchestra.run.trial040.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-13T06:36:28.889 INFO:teuthology.orchestra.run.trial040.stdout: re2-1:20211101-20.el9.x86_64 2026-02-13T06:36:28.889 INFO:teuthology.orchestra.run.trial040.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-13T06:36:28.889 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:28.889 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:36:28.929 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-test 2026-02-13T06:36:29.231 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:14 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:36:29.314 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:36:29.314 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repository Size 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout: ceph-test x86_64 2:20.2.0-636.g3b40640f.el9 ceph 84 M 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout:Installing dependencies: 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout: libcephsqlite x86_64 2:20.2.0-636.g3b40640f.el9 ceph 164 k 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-13T06:36:29.315 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:29.316 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:36:29.316 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:36:29.316 INFO:teuthology.orchestra.run.trial040.stdout:Install 5 Packages 2026-02-13T06:36:29.316 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:29.316 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 85 M 2026-02-13T06:36:29.316 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 364 M 2026-02-13T06:36:29.316 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:36:29.728 INFO:teuthology.orchestra.run.trial040.stdout:(1/5): libcephsqlite-20.2.0-636.g3b40640f.el9.x 467 kB/s | 164 kB 00:00 2026-02-13T06:36:30.102 INFO:teuthology.orchestra.run.trial040.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 321 kB/s | 233 kB 00:00 2026-02-13T06:36:30.478 INFO:teuthology.orchestra.run.trial040.stdout:(3/5): socat-1.7.4.1-8.el9.x86_64.rpm 404 kB/s | 303 kB 00:00 2026-02-13T06:36:30.692 INFO:teuthology.orchestra.run.trial040.stdout:(4/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 108 kB/s | 64 kB 00:00 2026-02-13T06:36:45.478 INFO:teuthology.orchestra.run.trial040.stdout:(5/5): ceph-test-20.2.0-636.g3b40640f.el9.x86_6 5.2 MB/s | 84 MB 00:16 2026-02-13T06:36:45.479 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:36:45.479 INFO:teuthology.orchestra.run.trial040.stdout:Total 5.2 MB/s | 85 MB 00:16 2026-02-13T06:36:45.502 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:36:45.512 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:36:45.512 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:36:45.800 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:36:45.800 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:36:46.114 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:36:46.121 INFO:teuthology.orchestra.run.trial040.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-13T06:36:46.126 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-13T06:36:46.134 INFO:teuthology.orchestra.run.trial040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-13T06:36:46.135 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libcephsqlite-2:20.2.0-636.g3b40640f.el9.x86_64 4/5 2026-02-13T06:36:49.871 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libcephsqlite-2:20.2.0-636.g3b40640f.el9.x86_64 4/5 2026-02-13T06:36:49.889 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-test-2:20.2.0-636.g3b40640f.el9.x86_64 5/5 2026-02-13T06:36:50.209 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-test-2:20.2.0-636.g3b40640f.el9.x86_64 5/5 2026-02-13T06:36:50.209 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-test-2:20.2.0-636.g3b40640f.el9.x86_64 1/5 2026-02-13T06:36:50.209 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libcephsqlite-2:20.2.0-636.g3b40640f.el9.x86_64 2/5 2026-02-13T06:36:50.209 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-13T06:36:50.209 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout: ceph-test-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout: libcephsqlite-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:50.265 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:36:50.302 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph 2026-02-13T06:36:50.600 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:35 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:36:50.686 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:36:50.688 INFO:teuthology.orchestra.run.trial040.stdout:====================================================================================== 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repository Size 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout:====================================================================================== 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: ceph x86_64 2:20.2.0-636.g3b40640f.el9 ceph 6.5 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout:Installing dependencies: 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: ceph-grafana-dashboards noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 43 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mds x86_64 2:20.2.0-636.g3b40640f.el9 ceph 2.3 M 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr x86_64 2:20.2.0-636.g3b40640f.el9 ceph 961 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-modules-core noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 290 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mon x86_64 2:20.2.0-636.g3b40640f.el9 ceph 5.0 M 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: ceph-osd x86_64 2:20.2.0-636.g3b40640f.el9 ceph 17 M 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: ceph-prometheus-alerts noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 17 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: cephadm noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 1.0 M 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-13T06:36:50.689 INFO:teuthology.orchestra.run.trial040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-13T06:36:50.690 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout:Installing weak dependencies: 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-cephadm noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 173 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-dashboard noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 11 M 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-diskprediction-local noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 7.4 M 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-k8sevents noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 22 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-rook noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 50 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: ceph-volume noarch 2:20.2.0-636.g3b40640f.el9 ceph-noarch 297 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout:====================================================================================== 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout:Install 95 Packages 2026-02-13T06:36:50.691 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:50.692 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 101 M 2026-02-13T06:36:50.692 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 561 M 2026-02-13T06:36:50.692 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:36:51.127 INFO:teuthology.orchestra.run.trial040.stdout:(1/95): ceph-20.2.0-636.g3b40640f.el9.x86_64.rp 46 kB/s | 6.5 kB 00:00 2026-02-13T06:36:51.204 INFO:teuthology.orchestra.run.trial040.stdout:(2/95): ceph-mgr-20.2.0-636.g3b40640f.el9.x86_6 4.3 MB/s | 961 kB 00:00 2026-02-13T06:36:51.218 INFO:teuthology.orchestra.run.trial040.stdout:(3/95): ceph-mds-20.2.0-636.g3b40640f.el9.x86_6 10 MB/s | 2.3 MB 00:00 2026-02-13T06:36:51.233 INFO:teuthology.orchestra.run.trial040.stdout:(4/95): ceph-grafana-dashboards-20.2.0-636.g3b4 2.8 MB/s | 43 kB 00:00 2026-02-13T06:36:51.252 INFO:teuthology.orchestra.run.trial040.stdout:(5/95): ceph-mgr-cephadm-20.2.0-636.g3b40640f.e 8.9 MB/s | 173 kB 00:00 2026-02-13T06:36:51.590 INFO:teuthology.orchestra.run.trial040.stdout:(6/95): ceph-mgr-dashboard-20.2.0-636.g3b40640f 31 MB/s | 11 MB 00:00 2026-02-13T06:36:51.836 INFO:teuthology.orchestra.run.trial040.stdout:(7/95): ceph-mgr-diskprediction-local-20.2.0-63 30 MB/s | 7.4 MB 00:00 2026-02-13T06:36:51.863 INFO:teuthology.orchestra.run.trial040.stdout:(8/95): ceph-osd-20.2.0-636.g3b40640f.el9.x86_6 26 MB/s | 17 MB 00:00 2026-02-13T06:36:51.863 INFO:teuthology.orchestra.run.trial040.stdout:(9/95): ceph-mgr-k8sevents-20.2.0-636.g3b40640f 829 kB/s | 22 kB 00:00 2026-02-13T06:36:51.875 INFO:teuthology.orchestra.run.trial040.stdout:(10/95): ceph-mgr-rook-20.2.0-636.g3b40640f.el9 4.2 MB/s | 50 kB 00:00 2026-02-13T06:36:51.876 INFO:teuthology.orchestra.run.trial040.stdout:(11/95): ceph-mgr-modules-core-20.2.0-636.g3b40 22 MB/s | 290 kB 00:00 2026-02-13T06:36:51.887 INFO:teuthology.orchestra.run.trial040.stdout:(12/95): ceph-prometheus-alerts-20.2.0-636.g3b4 1.4 MB/s | 17 kB 00:00 2026-02-13T06:36:51.888 INFO:teuthology.orchestra.run.trial040.stdout:(13/95): ceph-volume-20.2.0-636.g3b40640f.el9.n 24 MB/s | 297 kB 00:00 2026-02-13T06:36:51.906 INFO:teuthology.orchestra.run.trial040.stdout:(14/95): cephadm-20.2.0-636.g3b40640f.el9.noarc 51 MB/s | 1.0 MB 00:00 2026-02-13T06:36:51.910 INFO:teuthology.orchestra.run.trial040.stdout:(15/95): cryptsetup-2.8.1-2.el9.x86_64.rpm 16 MB/s | 350 kB 00:00 2026-02-13T06:36:51.915 INFO:teuthology.orchestra.run.trial040.stdout:(16/95): libconfig-1.7.2-9.el9.x86_64.rpm 15 MB/s | 72 kB 00:00 2026-02-13T06:36:51.917 INFO:teuthology.orchestra.run.trial040.stdout:(17/95): ledmon-libs-1.1.0-3.el9.x86_64.rpm 3.7 MB/s | 40 kB 00:00 2026-02-13T06:36:51.924 INFO:teuthology.orchestra.run.trial040.stdout:(18/95): libgfortran-11.5.0-14.el9.x86_64.rpm 83 MB/s | 794 kB 00:00 2026-02-13T06:36:51.929 INFO:teuthology.orchestra.run.trial040.stdout:(19/95): libquadmath-11.5.0-14.el9.x86_64.rpm 15 MB/s | 184 kB 00:00 2026-02-13T06:36:51.934 INFO:teuthology.orchestra.run.trial040.stdout:(20/95): python3-cffi-1.14.5-5.el9.x86_64.rpm 27 MB/s | 253 kB 00:00 2026-02-13T06:36:51.940 INFO:teuthology.orchestra.run.trial040.stdout:(21/95): python3-ply-3.11-14.el9.noarch.rpm 18 MB/s | 106 kB 00:00 2026-02-13T06:36:51.946 INFO:teuthology.orchestra.run.trial040.stdout:(22/95): python3-pycparser-2.20-6.el9.noarch.rp 21 MB/s | 135 kB 00:00 2026-02-13T06:36:51.953 INFO:teuthology.orchestra.run.trial040.stdout:(23/95): python3-pyparsing-2.4.7-9.el9.noarch.r 22 MB/s | 150 kB 00:00 2026-02-13T06:36:51.960 INFO:teuthology.orchestra.run.trial040.stdout:(24/95): python3-requests-2.25.1-10.el9.noarch. 18 MB/s | 126 kB 00:00 2026-02-13T06:36:51.965 INFO:teuthology.orchestra.run.trial040.stdout:(25/95): python3-cryptography-36.0.1-5.el9.x86_ 34 MB/s | 1.2 MB 00:00 2026-02-13T06:36:51.966 INFO:teuthology.orchestra.run.trial040.stdout:(26/95): python3-urllib3-1.26.5-7.el9.noarch.rp 32 MB/s | 218 kB 00:00 2026-02-13T06:36:51.972 INFO:teuthology.orchestra.run.trial040.stdout:(27/95): unzip-6.0-59.el9.x86_64.rpm 28 MB/s | 182 kB 00:00 2026-02-13T06:36:51.975 INFO:teuthology.orchestra.run.trial040.stdout:(28/95): zip-3.0-35.el9.x86_64.rpm 30 MB/s | 266 kB 00:00 2026-02-13T06:36:51.998 INFO:teuthology.orchestra.run.trial040.stdout:(29/95): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 33 kB 00:00 2026-02-13T06:36:52.007 INFO:teuthology.orchestra.run.trial040.stdout:(30/95): flexiblas-openblas-openmp-3.0.4-8.el9. 2.0 MB/s | 18 kB 00:00 2026-02-13T06:36:52.038 INFO:teuthology.orchestra.run.trial040.stdout:(31/95): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 7.8 MB/s | 246 kB 00:00 2026-02-13T06:36:52.050 INFO:teuthology.orchestra.run.trial040.stdout:(32/95): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 40 MB/s | 3.0 MB 00:00 2026-02-13T06:36:52.051 INFO:teuthology.orchestra.run.trial040.stdout:(33/95): lua-5.4.4-4.el9.x86_64.rpm 14 MB/s | 188 kB 00:00 2026-02-13T06:36:52.059 INFO:teuthology.orchestra.run.trial040.stdout:(34/95): openblas-0.3.29-1.el9.x86_64.rpm 4.7 MB/s | 42 kB 00:00 2026-02-13T06:36:52.067 INFO:teuthology.orchestra.run.trial040.stdout:(35/95): ceph-mon-20.2.0-636.g3b40640f.el9.x86_ 5.4 MB/s | 5.0 MB 00:00 2026-02-13T06:36:52.071 INFO:teuthology.orchestra.run.trial040.stdout:(36/95): protobuf-3.14.0-17.el9.x86_64.rpm 84 MB/s | 1.0 MB 00:00 2026-02-13T06:36:52.082 INFO:teuthology.orchestra.run.trial040.stdout:(37/95): python3-devel-3.9.25-2.el9.x86_64.rpm 21 MB/s | 244 kB 00:00 2026-02-13T06:36:52.097 INFO:teuthology.orchestra.run.trial040.stdout:(38/95): openblas-openmp-0.3.29-1.el9.x86_64.rp 116 MB/s | 5.3 MB 00:00 2026-02-13T06:36:52.098 INFO:teuthology.orchestra.run.trial040.stdout:(39/95): python3-jinja2-2.11.3-8.el9.noarch.rpm 16 MB/s | 249 kB 00:00 2026-02-13T06:36:52.107 INFO:teuthology.orchestra.run.trial040.stdout:(40/95): python3-jmespath-1.0.1-1.el9.noarch.rp 4.8 MB/s | 48 kB 00:00 2026-02-13T06:36:52.108 INFO:teuthology.orchestra.run.trial040.stdout:(41/95): python3-libstoragemgmt-1.10.1-1.el9.x8 16 MB/s | 177 kB 00:00 2026-02-13T06:36:52.117 INFO:teuthology.orchestra.run.trial040.stdout:(42/95): python3-markupsafe-1.1.1-12.el9.x86_64 3.8 MB/s | 35 kB 00:00 2026-02-13T06:36:52.120 INFO:teuthology.orchestra.run.trial040.stdout:(43/95): python3-lxml-4.6.5-3.el9.x86_64.rpm 90 MB/s | 1.2 MB 00:00 2026-02-13T06:36:52.133 INFO:teuthology.orchestra.run.trial040.stdout:(44/95): python3-numpy-f2py-1.23.5-2.el9.x86_64 35 MB/s | 442 kB 00:00 2026-02-13T06:36:52.143 INFO:teuthology.orchestra.run.trial040.stdout:(45/95): python3-packaging-20.9-5.el9.noarch.rp 7.5 MB/s | 77 kB 00:00 2026-02-13T06:36:52.153 INFO:teuthology.orchestra.run.trial040.stdout:(46/95): python3-protobuf-3.14.0-17.el9.noarch. 25 MB/s | 267 kB 00:00 2026-02-13T06:36:52.167 INFO:teuthology.orchestra.run.trial040.stdout:(47/95): python3-pyasn1-0.4.8-6.el9.noarch.rpm 11 MB/s | 159 kB 00:00 2026-02-13T06:36:52.182 INFO:teuthology.orchestra.run.trial040.stdout:(48/95): python3-babel-2.9.1-2.el9.noarch.rpm 52 MB/s | 6.0 MB 00:00 2026-02-13T06:36:52.183 INFO:teuthology.orchestra.run.trial040.stdout:(49/95): python3-pyasn1-modules-0.4.8-6.el9.noa 18 MB/s | 279 kB 00:00 2026-02-13T06:36:52.191 INFO:teuthology.orchestra.run.trial040.stdout:(50/95): python3-requests-oauthlib-1.3.0-12.el9 5.8 MB/s | 54 kB 00:00 2026-02-13T06:36:52.199 INFO:teuthology.orchestra.run.trial040.stdout:(51/95): python3-toml-0.10.2-6.el9.noarch.rpm 5.2 MB/s | 42 kB 00:00 2026-02-13T06:36:52.208 INFO:teuthology.orchestra.run.trial040.stdout:(52/95): xmlsec1-1.2.29-13.el9.x86_64.rpm 21 MB/s | 189 kB 00:00 2026-02-13T06:36:52.216 INFO:teuthology.orchestra.run.trial040.stdout:(53/95): python3-numpy-1.23.5-2.el9.x86_64.rpm 62 MB/s | 6.1 MB 00:00 2026-02-13T06:36:52.217 INFO:teuthology.orchestra.run.trial040.stdout:(54/95): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 9.7 MB/s | 90 kB 00:00 2026-02-13T06:36:52.282 INFO:teuthology.orchestra.run.trial040.stdout:(55/95): python3-scipy-1.9.3-2.el9.x86_64.rpm 195 MB/s | 19 MB 00:00 2026-02-13T06:36:52.282 INFO:teuthology.orchestra.run.trial040.stdout:(56/95): lua-devel-5.4.4-4.el9.x86_64.rpm 339 kB/s | 22 kB 00:00 2026-02-13T06:36:52.366 INFO:teuthology.orchestra.run.trial040.stdout:(57/95): protobuf-compiler-3.14.0-17.el9.x86_64 5.6 MB/s | 862 kB 00:00 2026-02-13T06:36:52.455 INFO:teuthology.orchestra.run.trial040.stdout:(58/95): grpc-data-1.46.7-10.el9.noarch.rpm 112 kB/s | 19 kB 00:00 2026-02-13T06:36:52.611 INFO:teuthology.orchestra.run.trial040.stdout:(59/95): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.6 MB/s | 551 kB 00:00 2026-02-13T06:36:52.612 INFO:teuthology.orchestra.run.trial040.stdout:(60/95): luarocks-3.9.2-5.el9.noarch.rpm 615 kB/s | 151 kB 00:00 2026-02-13T06:36:52.649 INFO:teuthology.orchestra.run.trial040.stdout:(61/95): python3-autocommand-2.2.2-8.el9.noarch 776 kB/s | 29 kB 00:00 2026-02-13T06:36:52.657 INFO:teuthology.orchestra.run.trial040.stdout:(62/95): python3-asyncssh-2.13.2-5.el9.noarch.r 2.6 MB/s | 548 kB 00:00 2026-02-13T06:36:52.659 INFO:teuthology.orchestra.run.trial040.stdout:(63/95): python3-backports-tarfile-1.2.0-1.el9. 1.3 MB/s | 60 kB 00:00 2026-02-13T06:36:52.688 INFO:teuthology.orchestra.run.trial040.stdout:(64/95): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.1 MB/s | 43 kB 00:00 2026-02-13T06:36:52.703 INFO:teuthology.orchestra.run.trial040.stdout:(65/95): python3-cachetools-4.2.4-1.el9.noarch. 713 kB/s | 32 kB 00:00 2026-02-13T06:36:52.711 INFO:teuthology.orchestra.run.trial040.stdout:(66/95): python3-certifi-2023.05.07-4.el9.noarc 274 kB/s | 14 kB 00:00 2026-02-13T06:36:52.733 INFO:teuthology.orchestra.run.trial040.stdout:(67/95): python3-cheroot-10.0.1-4.el9.noarch.rp 3.8 MB/s | 173 kB 00:00 2026-02-13T06:36:52.756 INFO:teuthology.orchestra.run.trial040.stdout:(68/95): python3-cherrypy-18.6.1-2.el9.noarch.r 6.5 MB/s | 358 kB 00:00 2026-02-13T06:36:52.801 INFO:teuthology.orchestra.run.trial040.stdout:(69/95): python3-google-auth-2.45.0-1.el9.noarc 2.7 MB/s | 254 kB 00:00 2026-02-13T06:36:52.808 INFO:teuthology.orchestra.run.trial040.stdout:(70/95): python3-grpcio-tools-1.46.7-10.el9.x86 2.7 MB/s | 144 kB 00:00 2026-02-13T06:36:52.854 INFO:teuthology.orchestra.run.trial040.stdout:(71/95): python3-grpcio-1.46.7-10.el9.x86_64.rp 17 MB/s | 2.0 MB 00:00 2026-02-13T06:36:52.868 INFO:teuthology.orchestra.run.trial040.stdout:(72/95): python3-isodate-0.6.1-3.el9.noarch.rpm 945 kB/s | 56 kB 00:00 2026-02-13T06:36:52.883 INFO:teuthology.orchestra.run.trial040.stdout:(73/95): python3-influxdb-5.3.1-1.el9.noarch.rp 1.7 MB/s | 139 kB 00:00 2026-02-13T06:36:52.891 INFO:teuthology.orchestra.run.trial040.stdout:(74/95): python3-jaraco-8.2.1-3.el9.noarch.rpm 283 kB/s | 11 kB 00:00 2026-02-13T06:36:52.906 INFO:teuthology.orchestra.run.trial040.stdout:(75/95): python3-jaraco-classes-3.2.1-5.el9.noa 465 kB/s | 18 kB 00:00 2026-02-13T06:36:52.922 INFO:teuthology.orchestra.run.trial040.stdout:(76/95): python3-jaraco-collections-3.0.0-8.el9 605 kB/s | 23 kB 00:00 2026-02-13T06:36:52.930 INFO:teuthology.orchestra.run.trial040.stdout:(77/95): python3-jaraco-context-6.0.1-3.el9.noa 517 kB/s | 20 kB 00:00 2026-02-13T06:36:52.944 INFO:teuthology.orchestra.run.trial040.stdout:(78/95): python3-jaraco-functools-3.5.0-2.el9.n 509 kB/s | 19 kB 00:00 2026-02-13T06:36:52.960 INFO:teuthology.orchestra.run.trial040.stdout:(79/95): python3-jaraco-text-4.0.0-2.el9.noarch 693 kB/s | 26 kB 00:00 2026-02-13T06:36:52.984 INFO:teuthology.orchestra.run.trial040.stdout:(80/95): python3-more-itertools-8.12.0-2.el9.no 1.9 MB/s | 79 kB 00:00 2026-02-13T06:36:52.987 INFO:teuthology.orchestra.run.trial040.stdout:(81/95): python3-kubernetes-26.1.0-3.el9.noarch 18 MB/s | 1.0 MB 00:00 2026-02-13T06:36:53.013 INFO:teuthology.orchestra.run.trial040.stdout:(82/95): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.6 MB/s | 86 kB 00:00 2026-02-13T06:36:53.024 INFO:teuthology.orchestra.run.trial040.stdout:(83/95): python3-portend-3.1.0-2.el9.noarch.rpm 436 kB/s | 16 kB 00:00 2026-02-13T06:36:53.028 INFO:teuthology.orchestra.run.trial040.stdout:(84/95): python3-natsort-7.1.1-5.el9.noarch.rpm 1.3 MB/s | 58 kB 00:00 2026-02-13T06:36:53.055 INFO:teuthology.orchestra.run.trial040.stdout:(85/95): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2026-02-13T06:36:53.067 INFO:teuthology.orchestra.run.trial040.stdout:(86/95): python3-repoze-lru-0.7-16.el9.noarch.r 715 kB/s | 31 kB 00:00 2026-02-13T06:36:53.081 INFO:teuthology.orchestra.run.trial040.stdout:(87/95): python3-routes-2.5.1-5.el9.noarch.rpm 3.5 MB/s | 188 kB 00:00 2026-02-13T06:36:53.100 INFO:teuthology.orchestra.run.trial040.stdout:(88/95): python3-rsa-4.9-2.el9.noarch.rpm 1.3 MB/s | 59 kB 00:00 2026-02-13T06:36:53.115 INFO:teuthology.orchestra.run.trial040.stdout:(89/95): python3-saml-1.16.0-1.el9.noarch.rpm 2.5 MB/s | 125 kB 00:00 2026-02-13T06:36:53.119 INFO:teuthology.orchestra.run.trial040.stdout:(90/95): python3-tempora-5.0.0-2.el9.noarch.rpm 936 kB/s | 36 kB 00:00 2026-02-13T06:36:53.152 INFO:teuthology.orchestra.run.trial040.stdout:(91/95): python3-typing-extensions-4.15.0-1.el9 1.6 MB/s | 86 kB 00:00 2026-02-13T06:36:53.161 INFO:teuthology.orchestra.run.trial040.stdout:(92/95): python3-websocket-client-1.2.3-2.el9.n 1.9 MB/s | 90 kB 00:00 2026-02-13T06:36:53.171 INFO:teuthology.orchestra.run.trial040.stdout:(93/95): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 941 kB/s | 48 kB 00:00 2026-02-13T06:36:53.196 INFO:teuthology.orchestra.run.trial040.stdout:(94/95): python3-xmltodict-0.12.0-15.el9.noarch 505 kB/s | 22 kB 00:00 2026-02-13T06:36:53.199 INFO:teuthology.orchestra.run.trial040.stdout:(95/95): python3-zc-lockfile-2.0-10.el9.noarch. 528 kB/s | 20 kB 00:00 2026-02-13T06:36:53.201 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:36:53.201 INFO:teuthology.orchestra.run.trial040.stdout:Total 40 MB/s | 101 MB 00:02 2026-02-13T06:36:53.643 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:36:53.681 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:36:53.681 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:36:54.114 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:36:54.114 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:36:54.585 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:36:54.587 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/95 2026-02-13T06:36:54.591 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/95 2026-02-13T06:36:54.613 INFO:teuthology.orchestra.run.trial040.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/95 2026-02-13T06:36:54.646 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/95 2026-02-13T06:36:54.655 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/95 2026-02-13T06:36:54.660 INFO:teuthology.orchestra.run.trial040.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 6/95 2026-02-13T06:36:54.663 INFO:teuthology.orchestra.run.trial040.stdout: Installing : lua-5.4.4-4.el9.x86_64 7/95 2026-02-13T06:36:54.667 INFO:teuthology.orchestra.run.trial040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 8/95 2026-02-13T06:36:54.690 INFO:teuthology.orchestra.run.trial040.stdout: Installing : unzip-6.0-59.el9.x86_64 9/95 2026-02-13T06:36:54.703 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 10/95 2026-02-13T06:36:54.706 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 11/95 2026-02-13T06:36:54.712 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 12/95 2026-02-13T06:36:54.714 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 13/95 2026-02-13T06:36:54.725 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 14/95 2026-02-13T06:36:54.731 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 15/95 2026-02-13T06:36:54.755 INFO:teuthology.orchestra.run.trial040.stdout: Installing : zip-3.0-35.el9.x86_64 16/95 2026-02-13T06:36:54.758 INFO:teuthology.orchestra.run.trial040.stdout: Installing : luarocks-3.9.2-5.el9.noarch 17/95 2026-02-13T06:36:54.766 INFO:teuthology.orchestra.run.trial040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 18/95 2026-02-13T06:36:54.816 INFO:teuthology.orchestra.run.trial040.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 19/95 2026-02-13T06:36:54.832 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 20/95 2026-02-13T06:36:54.835 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 21/95 2026-02-13T06:36:54.839 INFO:teuthology.orchestra.run.trial040.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 22/95 2026-02-13T06:36:54.844 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 23/95 2026-02-13T06:36:54.849 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 24/95 2026-02-13T06:36:54.852 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 25/95 2026-02-13T06:36:54.867 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 26/95 2026-02-13T06:36:54.871 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 27/95 2026-02-13T06:36:54.876 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 28/95 2026-02-13T06:36:54.888 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 29/95 2026-02-13T06:36:54.897 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 30/95 2026-02-13T06:36:54.903 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 31/95 2026-02-13T06:36:54.923 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 32/95 2026-02-13T06:36:54.934 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 33/95 2026-02-13T06:36:54.949 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 34/95 2026-02-13T06:36:54.954 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 35/95 2026-02-13T06:36:54.961 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 36/95 2026-02-13T06:36:55.001 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 37/95 2026-02-13T06:36:55.325 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 38/95 2026-02-13T06:36:55.335 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 39/95 2026-02-13T06:36:55.340 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 40/95 2026-02-13T06:36:55.346 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 41/95 2026-02-13T06:36:55.347 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 42/95 2026-02-13T06:36:55.376 INFO:teuthology.orchestra.run.trial040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 43/95 2026-02-13T06:36:55.419 INFO:teuthology.orchestra.run.trial040.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 44/95 2026-02-13T06:36:55.426 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 45/95 2026-02-13T06:36:55.430 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 46/95 2026-02-13T06:36:55.435 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 47/95 2026-02-13T06:36:55.439 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 48/95 2026-02-13T06:36:55.446 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 49/95 2026-02-13T06:36:55.450 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 50/95 2026-02-13T06:36:55.477 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 51/95 2026-02-13T06:36:55.487 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 52/95 2026-02-13T06:36:55.493 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 53/95 2026-02-13T06:36:55.500 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 54/95 2026-02-13T06:36:55.535 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 55/95 2026-02-13T06:36:55.758 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 56/95 2026-02-13T06:36:55.782 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 57/95 2026-02-13T06:36:55.786 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 58/95 2026-02-13T06:36:55.844 INFO:teuthology.orchestra.run.trial040.stdout: Installing : openblas-0.3.29-1.el9.x86_64 59/95 2026-02-13T06:36:55.846 INFO:teuthology.orchestra.run.trial040.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 60/95 2026-02-13T06:36:55.867 INFO:teuthology.orchestra.run.trial040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 61/95 2026-02-13T06:36:56.192 INFO:teuthology.orchestra.run.trial040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 62/95 2026-02-13T06:36:56.265 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 63/95 2026-02-13T06:36:56.934 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 64/95 2026-02-13T06:36:56.951 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 65/95 2026-02-13T06:36:56.961 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 66/95 2026-02-13T06:36:56.968 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/95 2026-02-13T06:36:56.981 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-ply-3.11-14.el9.noarch 68/95 2026-02-13T06:36:56.996 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 69/95 2026-02-13T06:36:57.071 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 70/95 2026-02-13T06:36:57.080 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 71/95 2026-02-13T06:36:57.103 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 72/95 2026-02-13T06:36:57.134 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 73/95 2026-02-13T06:36:57.185 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 74/95 2026-02-13T06:36:57.191 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 75/95 2026-02-13T06:36:57.194 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/95 2026-02-13T06:36:57.195 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 77/95 2026-02-13T06:36:57.214 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-13T06:36:57.214 INFO:teuthology.orchestra.run.trial040.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-13T06:36:57.214 INFO:teuthology.orchestra.run.trial040.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-13T06:36:57.214 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:57.223 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-13T06:36:57.250 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-13T06:36:57.250 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-13T06:36:57.250 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:57.286 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 79/95 2026-02-13T06:36:57.350 INFO:teuthology.orchestra.run.trial040.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 80/95 2026-02-13T06:36:57.352 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-volume-2:20.2.0-636.g3b40640f.el9.noarch 81/95 2026-02-13T06:36:57.366 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-volume-2:20.2.0-636.g3b40640f.el9.noarch 81/95 2026-02-13T06:36:57.366 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T06:36:57.366 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-13T06:36:57.366 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:58.043 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-osd-2:20.2.0-636.g3b40640f.el9.x86_64 82/95 2026-02-13T06:36:58.069 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-osd-2:20.2.0-636.g3b40640f.el9.x86_64 82/95 2026-02-13T06:36:58.069 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T06:36:58.069 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-13T06:36:58.070 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T06:36:58.070 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T06:36:58.070 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:58.130 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: cephadm-2:20.2.0-636.g3b40640f.el9.noarch 83/95 2026-02-13T06:36:58.132 INFO:teuthology.orchestra.run.trial040.stdout: Installing : cephadm-2:20.2.0-636.g3b40640f.el9.noarch 83/95 2026-02-13T06:36:58.139 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-prometheus-alerts-2:20.2.0-636.g3b40640f.el9. 84/95 2026-02-13T06:36:58.162 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-grafana-dashboards-2:20.2.0-636.g3b40640f.el9 85/95 2026-02-13T06:36:58.163 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-cephadm-2:20.2.0-636.g3b40640f.el9.noarch 86/95 2026-02-13T06:36:59.319 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.2.0-636.g3b40640f.el9.noarch 86/95 2026-02-13T06:36:59.324 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-dashboard-2:20.2.0-636.g3b40640f.el9.noar 87/95 2026-02-13T06:36:59.784 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.2.0-636.g3b40640f.el9.noar 87/95 2026-02-13T06:36:59.785 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-diskprediction-local-2:20.2.0-636.g3b4064 88/95 2026-02-13T06:36:59.797 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.2.0-636.g3b4064 88/95 2026-02-13T06:36:59.798 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-k8sevents-2:20.2.0-636.g3b40640f.el9.noar 89/95 2026-02-13T06:36:59.855 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.2.0-636.g3b40640f.el9.noar 89/95 2026-02-13T06:36:59.897 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-modules-core-2:20.2.0-636.g3b40640f.el9.n 90/95 2026-02-13T06:36:59.898 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-2:20.2.0-636.g3b40640f.el9.x86_64 91/95 2026-02-13T06:36:59.922 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-2:20.2.0-636.g3b40640f.el9.x86_64 91/95 2026-02-13T06:36:59.922 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T06:36:59.922 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-13T06:36:59.922 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T06:36:59.922 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T06:36:59.922 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:36:59.935 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-rook-2:20.2.0-636.g3b40640f.el9.noarch 92/95 2026-02-13T06:37:00.162 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-rook-2:20.2.0-636.g3b40640f.el9.noarch 92/95 2026-02-13T06:37:00.163 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mon-2:20.2.0-636.g3b40640f.el9.x86_64 93/95 2026-02-13T06:37:00.186 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mon-2:20.2.0-636.g3b40640f.el9.x86_64 93/95 2026-02-13T06:37:00.186 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T06:37:00.186 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-13T06:37:00.186 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T06:37:00.186 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T06:37:00.186 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:00.283 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mds-2:20.2.0-636.g3b40640f.el9.x86_64 94/95 2026-02-13T06:37:00.303 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mds-2:20.2.0-636.g3b40640f.el9.x86_64 94/95 2026-02-13T06:37:00.303 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T06:37:00.303 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-13T06:37:00.303 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T06:37:00.304 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T06:37:00.304 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:00.321 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-2:20.2.0-636.g3b40640f.el9.x86_64 95/95 2026-02-13T06:37:01.049 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-2:20.2.0-636.g3b40640f.el9.x86_64 95/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-2:20.2.0-636.g3b40640f.el9.x86_64 1/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mds-2:20.2.0-636.g3b40640f.el9.x86_64 2/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-2:20.2.0-636.g3b40640f.el9.x86_64 3/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mon-2:20.2.0-636.g3b40640f.el9.x86_64 4/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-osd-2:20.2.0-636.g3b40640f.el9.x86_64 5/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-grafana-dashboards-2:20.2.0-636.g3b40640f.el9 6/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-cephadm-2:20.2.0-636.g3b40640f.el9.noarch 7/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-dashboard-2:20.2.0-636.g3b40640f.el9.noar 8/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.2.0-636.g3b4064 9/95 2026-02-13T06:37:01.050 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-k8sevents-2:20.2.0-636.g3b40640f.el9.noar 10/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-modules-core-2:20.2.0-636.g3b40640f.el9.n 11/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-rook-2:20.2.0-636.g3b40640f.el9.noarch 12/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-prometheus-alerts-2:20.2.0-636.g3b40640f.el9. 13/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-volume-2:20.2.0-636.g3b40640f.el9.noarch 14/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : cephadm-2:20.2.0-636.g3b40640f.el9.noarch 15/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 16/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/95 2026-02-13T06:37:01.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : zip-3.0-35.el9.x86_64 29/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 30/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 31/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 32/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/95 2026-02-13T06:37:01.052 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 39/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 41/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 42/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 43/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/95 2026-02-13T06:37:01.053 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/95 2026-02-13T06:37:01.054 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 81/95 2026-02-13T06:37:01.055 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 82/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 83/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 84/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 85/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 86/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 87/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 88/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 89/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 90/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 91/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 92/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 93/95 2026-02-13T06:37:01.056 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 94/95 2026-02-13T06:37:01.113 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 95/95 2026-02-13T06:37:01.113 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:01.113 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:37:01.113 INFO:teuthology.orchestra.run.trial040.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-13T06:37:01.113 INFO:teuthology.orchestra.run.trial040.stdout: ceph-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:01.113 INFO:teuthology.orchestra.run.trial040.stdout: ceph-grafana-dashboards-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.113 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mds-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:01.113 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:01.113 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-cephadm-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-dashboard-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-diskprediction-local-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-k8sevents-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-modules-core-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-rook-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mon-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: ceph-osd-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: ceph-prometheus-alerts-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: ceph-volume-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: cephadm-2:20.2.0-636.g3b40640f.el9.noarch 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-13T06:37:01.114 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: lua-5.4.4-4.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-13T06:37:01.115 INFO:teuthology.orchestra.run.trial040.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-13T06:37:01.116 INFO:teuthology.orchestra.run.trial040.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-13T06:37:01.117 INFO:teuthology.orchestra.run.trial040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-ply-3.11-14.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-13T06:37:01.118 INFO:teuthology.orchestra.run.trial040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-13T06:37:01.119 INFO:teuthology.orchestra.run.trial040.stdout: unzip-6.0-59.el9.x86_64 2026-02-13T06:37:01.120 INFO:teuthology.orchestra.run.trial040.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-13T06:37:01.120 INFO:teuthology.orchestra.run.trial040.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-13T06:37:01.120 INFO:teuthology.orchestra.run.trial040.stdout: zip-3.0-35.el9.x86_64 2026-02-13T06:37:01.120 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:01.120 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:01.168 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-base 2026-02-13T06:37:01.469 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:46 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:01.534 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-base-2:20.2.0-636.g3b40640f.el9.x86_64 is already installed. 2026-02-13T06:37:01.552 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:01.553 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:01.553 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:01.573 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install cephadm 2026-02-13T06:37:01.870 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:46 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:01.934 INFO:teuthology.orchestra.run.trial040.stdout:Package cephadm-2:20.2.0-636.g3b40640f.el9.noarch is already installed. 2026-02-13T06:37:01.952 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:01.953 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:01.953 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:01.974 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-immutable-object-cache 2026-02-13T06:37:02.277 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:47 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:02.359 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:02.360 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:02.360 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout: ceph-immutable-object-cache x86_64 2:20.2.0-636.g3b40640f.el9 ceph 154 k 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 154 k 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 447 k 2026-02-13T06:37:02.361 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:37:02.721 INFO:teuthology.orchestra.run.trial040.stdout:ceph-immutable-object-cache-20.2.0-636.g3b40640 427 kB/s | 154 kB 00:00 2026-02-13T06:37:02.723 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:37:02.723 INFO:teuthology.orchestra.run.trial040.stdout:Total 424 kB/s | 154 kB 00:00 2026-02-13T06:37:02.723 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:37:02.729 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:37:02.729 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:37:02.761 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:37:02.761 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:37:02.817 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:37:02.818 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-immutable-object-cache-2:20.2.0-636.g3b40640f.e 1/1 2026-02-13T06:37:02.842 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.2.0-636.g3b40640f.e 1/1 2026-02-13T06:37:02.842 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T06:37:02.842 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-13T06:37:02.842 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:03.126 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-immutable-object-cache-2:20.2.0-636.g3b40640f.e 1/1 2026-02-13T06:37:03.126 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:03.126 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:37:03.126 INFO:teuthology.orchestra.run.trial040.stdout: ceph-immutable-object-cache-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:03.126 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:03.126 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:03.153 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr 2026-02-13T06:37:03.455 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:48 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:03.519 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-2:20.2.0-636.g3b40640f.el9.x86_64 is already installed. 2026-02-13T06:37:03.537 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:03.538 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:03.538 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:03.559 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr-dashboard 2026-02-13T06:37:03.859 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:48 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:03.924 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-dashboard-2:20.2.0-636.g3b40640f.el9.noarch is already installed. 2026-02-13T06:37:03.942 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:03.942 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:03.942 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:03.963 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-13T06:37:04.261 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:04.326 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-diskprediction-local-2:20.2.0-636.g3b40640f.el9.noarch is already installed. 2026-02-13T06:37:04.344 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:04.345 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:04.345 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:04.366 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr-rook 2026-02-13T06:37:04.663 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:04.729 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-rook-2:20.2.0-636.g3b40640f.el9.noarch is already installed. 2026-02-13T06:37:04.747 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:04.748 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:04.748 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:04.769 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr-cephadm 2026-02-13T06:37:05.067 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:05.132 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-cephadm-2:20.2.0-636.g3b40640f.el9.noarch is already installed. 2026-02-13T06:37:05.150 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:05.150 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:05.151 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:05.171 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-fuse 2026-02-13T06:37:05.471 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:05.554 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:05.555 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:05.555 INFO:teuthology.orchestra.run.trial040.stdout: Package Architecture Version Repository Size 2026-02-13T06:37:05.555 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:05.555 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:37:05.555 INFO:teuthology.orchestra.run.trial040.stdout: ceph-fuse x86_64 2:20.2.0-636.g3b40640f.el9 ceph 941 k 2026-02-13T06:37:05.555 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:05.556 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:37:05.556 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:05.556 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-13T06:37:05.556 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:05.556 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 941 k 2026-02-13T06:37:05.556 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 2.7 M 2026-02-13T06:37:05.556 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:37:06.577 INFO:teuthology.orchestra.run.trial040.stdout:ceph-fuse-20.2.0-636.g3b40640f.el9.x86_64.rpm 922 kB/s | 941 kB 00:01 2026-02-13T06:37:06.579 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:37:06.579 INFO:teuthology.orchestra.run.trial040.stdout:Total 920 kB/s | 941 kB 00:01 2026-02-13T06:37:06.579 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:37:06.584 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:37:06.584 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:37:06.619 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:37:06.619 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:37:06.708 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:37:06.726 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-fuse-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:06.957 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-fuse-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:07.015 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-fuse-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:07.015 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:07.015 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:37:07.015 INFO:teuthology.orchestra.run.trial040.stdout: ceph-fuse-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:07.015 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:07.015 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:07.043 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-volume 2026-02-13T06:37:07.353 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:52 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:07.419 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-volume-2:20.2.0-636.g3b40640f.el9.noarch is already installed. 2026-02-13T06:37:07.437 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:07.437 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:07.438 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:07.460 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install librados-devel 2026-02-13T06:37:07.759 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:52 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:07.841 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:07.842 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:07.842 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-13T06:37:07.842 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:07.842 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:37:07.843 INFO:teuthology.orchestra.run.trial040.stdout: librados-devel x86_64 2:20.2.0-636.g3b40640f.el9 ceph 126 k 2026-02-13T06:37:07.843 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:07.843 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:37:07.843 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:07.843 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-13T06:37:07.843 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:07.843 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 126 k 2026-02-13T06:37:07.843 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 449 k 2026-02-13T06:37:07.843 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:37:08.204 INFO:teuthology.orchestra.run.trial040.stdout:librados-devel-20.2.0-636.g3b40640f.el9.x86_64. 348 kB/s | 126 kB 00:00 2026-02-13T06:37:08.205 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:37:08.205 INFO:teuthology.orchestra.run.trial040.stdout:Total 346 kB/s | 126 kB 00:00 2026-02-13T06:37:08.205 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:37:08.208 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:37:08.208 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:37:08.240 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:37:08.240 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:37:08.295 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:37:08.311 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librados-devel-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:08.386 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librados-devel-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:08.443 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librados-devel-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:08.444 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:08.444 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:37:08.444 INFO:teuthology.orchestra.run.trial040.stdout: librados-devel-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:08.444 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:08.444 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:08.470 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install libcephfs2 2026-02-13T06:37:08.779 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:53 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:08.844 INFO:teuthology.orchestra.run.trial040.stdout:Package libcephfs2-2:20.2.0-636.g3b40640f.el9.x86_64 is already installed. 2026-02-13T06:37:08.862 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:08.862 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:08.863 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:08.884 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install libcephfs-devel 2026-02-13T06:37:09.188 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:54 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:09.270 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:09.271 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:09.271 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-13T06:37:09.271 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:09.271 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:37:09.271 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs-devel x86_64 2:20.2.0-636.g3b40640f.el9 ceph 34 k 2026-02-13T06:37:09.271 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:09.271 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:37:09.271 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:09.271 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-13T06:37:09.272 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:09.272 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 34 k 2026-02-13T06:37:09.272 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 155 k 2026-02-13T06:37:09.272 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:37:09.528 INFO:teuthology.orchestra.run.trial040.stdout:libcephfs-devel-20.2.0-636.g3b40640f.el9.x86_64 133 kB/s | 34 kB 00:00 2026-02-13T06:37:09.529 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:37:09.529 INFO:teuthology.orchestra.run.trial040.stdout:Total 133 kB/s | 34 kB 00:00 2026-02-13T06:37:09.529 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:37:09.532 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:37:09.532 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:37:09.534 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:37:09.535 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:37:09.556 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:37:09.572 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libcephfs-devel-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:09.626 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libcephfs-devel-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:09.683 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libcephfs-devel-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:09.683 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:09.683 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:37:09.683 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs-devel-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:09.683 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:09.683 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:09.710 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install librados2 2026-02-13T06:37:10.012 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:10.078 INFO:teuthology.orchestra.run.trial040.stdout:Package librados2-2:20.2.0-636.g3b40640f.el9.x86_64 is already installed. 2026-02-13T06:37:10.097 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:10.097 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:10.098 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:10.118 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install librbd1 2026-02-13T06:37:10.419 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:10.484 INFO:teuthology.orchestra.run.trial040.stdout:Package librbd1-2:20.2.0-636.g3b40640f.el9.x86_64 is already installed. 2026-02-13T06:37:10.502 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:10.503 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:10.503 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:10.524 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-rados 2026-02-13T06:37:10.824 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:10.889 INFO:teuthology.orchestra.run.trial040.stdout:Package python3-rados-2:20.2.0-636.g3b40640f.el9.x86_64 is already installed. 2026-02-13T06:37:10.907 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:10.908 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:10.908 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:10.929 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-rgw 2026-02-13T06:37:11.230 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:56 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:11.295 INFO:teuthology.orchestra.run.trial040.stdout:Package python3-rgw-2:20.2.0-636.g3b40640f.el9.x86_64 is already installed. 2026-02-13T06:37:11.314 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:11.314 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:11.314 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:11.336 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-cephfs 2026-02-13T06:37:11.636 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:56 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:11.700 INFO:teuthology.orchestra.run.trial040.stdout:Package python3-cephfs-2:20.2.0-636.g3b40640f.el9.x86_64 is already installed. 2026-02-13T06:37:11.718 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:11.719 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:11.719 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:11.740 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-rbd 2026-02-13T06:37:12.045 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:57 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:12.110 INFO:teuthology.orchestra.run.trial040.stdout:Package python3-rbd-2:20.2.0-636.g3b40640f.el9.x86_64 is already installed. 2026-02-13T06:37:12.128 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:12.129 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-13T06:37:12.129 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:12.151 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install rbd-fuse 2026-02-13T06:37:12.456 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:57 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:12.539 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:12.540 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:12.540 INFO:teuthology.orchestra.run.trial040.stdout: Package Architecture Version Repository Size 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout: rbd-fuse x86_64 2:20.2.0-636.g3b40640f.el9 ceph 91 k 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 91 k 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 238 k 2026-02-13T06:37:12.541 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:37:12.850 INFO:teuthology.orchestra.run.trial040.stdout:rbd-fuse-20.2.0-636.g3b40640f.el9.x86_64.rpm 295 kB/s | 91 kB 00:00 2026-02-13T06:37:12.851 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:37:12.851 INFO:teuthology.orchestra.run.trial040.stdout:Total 294 kB/s | 91 kB 00:00 2026-02-13T06:37:12.852 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:37:12.856 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:37:12.856 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:37:12.888 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:37:12.888 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:37:12.940 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:37:12.956 INFO:teuthology.orchestra.run.trial040.stdout: Installing : rbd-fuse-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:13.034 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: rbd-fuse-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:13.091 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : rbd-fuse-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:13.092 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:13.092 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:37:13.092 INFO:teuthology.orchestra.run.trial040.stdout: rbd-fuse-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:13.092 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:13.092 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:13.120 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install rbd-mirror 2026-02-13T06:37:13.428 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:58 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:13.511 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:13.512 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:13.512 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-13T06:37:13.512 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:13.512 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:37:13.512 INFO:teuthology.orchestra.run.trial040.stdout: rbd-mirror x86_64 2:20.2.0-636.g3b40640f.el9 ceph 2.9 M 2026-02-13T06:37:13.512 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:13.512 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:37:13.512 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:13.513 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-13T06:37:13.513 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:13.513 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 2.9 M 2026-02-13T06:37:13.513 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 11 M 2026-02-13T06:37:13.513 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:37:14.094 INFO:teuthology.orchestra.run.trial040.stdout:rbd-mirror-20.2.0-636.g3b40640f.el9.x86_64.rpm 5.0 MB/s | 2.9 MB 00:00 2026-02-13T06:37:14.095 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:37:14.095 INFO:teuthology.orchestra.run.trial040.stdout:Total 5.0 MB/s | 2.9 MB 00:00 2026-02-13T06:37:14.095 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:37:14.100 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:37:14.101 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:37:14.141 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:37:14.141 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:37:14.328 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:37:14.330 INFO:teuthology.orchestra.run.trial040.stdout: Installing : rbd-mirror-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:14.352 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: rbd-mirror-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:14.352 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T06:37:14.352 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-13T06:37:14.352 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T06:37:14.352 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T06:37:14.352 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:14.634 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : rbd-mirror-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:14.634 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:14.634 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:37:14.634 INFO:teuthology.orchestra.run.trial040.stdout: rbd-mirror-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:14.634 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:14.635 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:14.662 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install rbd-nbd 2026-02-13T06:37:14.970 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Feb 2026 06:36:15 AM UTC. 2026-02-13T06:37:15.053 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-13T06:37:15.054 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:15.054 INFO:teuthology.orchestra.run.trial040.stdout: Package Architecture Version Repository Size 2026-02-13T06:37:15.054 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:15.054 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-13T06:37:15.054 INFO:teuthology.orchestra.run.trial040.stdout: rbd-nbd x86_64 2:20.2.0-636.g3b40640f.el9 ceph 180 k 2026-02-13T06:37:15.054 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:15.054 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-13T06:37:15.054 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-13T06:37:15.054 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-13T06:37:15.055 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:15.055 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 180 k 2026-02-13T06:37:15.055 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 498 k 2026-02-13T06:37:15.055 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-13T06:37:15.409 INFO:teuthology.orchestra.run.trial040.stdout:rbd-nbd-20.2.0-636.g3b40640f.el9.x86_64.rpm 505 kB/s | 180 kB 00:00 2026-02-13T06:37:15.410 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-13T06:37:15.410 INFO:teuthology.orchestra.run.trial040.stdout:Total 504 kB/s | 180 kB 00:00 2026-02-13T06:37:15.411 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-13T06:37:15.415 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-13T06:37:15.416 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-13T06:37:15.448 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-13T06:37:15.448 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-13T06:37:15.504 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-13T06:37:15.521 INFO:teuthology.orchestra.run.trial040.stdout: Installing : rbd-nbd-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:15.604 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: rbd-nbd-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:15.662 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : rbd-nbd-2:20.2.0-636.g3b40640f.el9.x86_64 1/1 2026-02-13T06:37:15.662 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:15.662 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-13T06:37:15.662 INFO:teuthology.orchestra.run.trial040.stdout: rbd-nbd-2:20.2.0-636.g3b40640f.el9.x86_64 2026-02-13T06:37:15.662 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-13T06:37:15.662 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-13T06:37:15.693 DEBUG:teuthology.parallel:result is None 2026-02-13T06:37:15.694 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb 2026-02-13T06:37:15.876 DEBUG:teuthology.orchestra.run.trial040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-13T06:37:15.899 INFO:teuthology.orchestra.run.trial040.stdout:20.2.0-636.g3b40640f.el9 2026-02-13T06:37:15.900 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-636.g3b40640f.el9 2026-02-13T06:37:15.900 INFO:teuthology.task.install:The correct ceph version 20.2.0-636.g3b40640f is installed. 2026-02-13T06:37:15.903 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-13T06:37:15.903 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-13T06:37:15.904 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-13T06:37:15.964 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-13T06:37:15.964 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-13T06:37:15.964 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/usr/bin/daemon-helper 2026-02-13T06:37:16.028 DEBUG:teuthology.orchestra.run.trial040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-13T06:37:16.091 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-13T06:37:16.092 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-13T06:37:16.092 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-13T06:37:16.156 DEBUG:teuthology.orchestra.run.trial040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-13T06:37:16.220 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-13T06:37:16.220 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-13T06:37:16.220 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/usr/bin/stdin-killer 2026-02-13T06:37:16.284 DEBUG:teuthology.orchestra.run.trial040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-13T06:37:16.349 INFO:teuthology.run_tasks:Running task workunit... 2026-02-13T06:37:16.377 INFO:tasks.workunit:Pulling workunits from ref 3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb 2026-02-13T06:37:16.378 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-13T06:37:16.378 INFO:tasks.workunit:timeout=3h 2026-02-13T06:37:16.378 INFO:tasks.workunit:cleanup=True 2026-02-13T06:37:16.379 DEBUG:teuthology.orchestra.run.trial040:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-13T06:37:16.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T06:37:16.406 INFO:teuthology.orchestra.run.trial040.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-13T06:37:16.406 DEBUG:teuthology.orchestra.run.trial040:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-13T06:37:16.463 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-13T06:37:16.464 DEBUG:teuthology.orchestra.run.trial040:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-13T06:37:16.520 DEBUG:teuthology.orchestra.run.trial040:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb 2026-02-13T06:37:16.583 INFO:tasks.workunit.client.0.trial040.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-13T06:37:36.538 INFO:tasks.workunit.client.0.trial040.stderr:Note: switching to '3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb'. 2026-02-13T06:37:36.538 INFO:tasks.workunit.client.0.trial040.stderr: 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr:state without impacting any branches by switching back to a branch. 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr: 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr: 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr: git switch -c 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr: 2026-02-13T06:37:36.539 INFO:tasks.workunit.client.0.trial040.stderr:Or undo this operation with: 2026-02-13T06:37:36.540 INFO:tasks.workunit.client.0.trial040.stderr: 2026-02-13T06:37:36.540 INFO:tasks.workunit.client.0.trial040.stderr: git switch - 2026-02-13T06:37:36.540 INFO:tasks.workunit.client.0.trial040.stderr: 2026-02-13T06:37:36.540 INFO:tasks.workunit.client.0.trial040.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-13T06:37:36.540 INFO:tasks.workunit.client.0.trial040.stderr: 2026-02-13T06:37:36.540 INFO:tasks.workunit.client.0.trial040.stderr:HEAD is now at 3b40640f18f Merge branch 'wip-74888-tentacle' of https://github.com/aainscow/ceph into wip-bharath17-testing-2026-02-12-0618-tentacle 2026-02-13T06:37:36.547 DEBUG:teuthology.orchestra.run.trial040:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-13T06:37:36.605 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-13T06:37:36.605 DEBUG:teuthology.orchestra.run.trial040:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-13T06:37:36.663 INFO:tasks.workunit:Running workunits matching misc on client.0... 2026-02-13T06:37:36.664 INFO:tasks.workunit:Running workunit misc/mclock-config.sh... 2026-02-13T06:37:36.664 DEBUG:teuthology.orchestra.run.trial040:workunit test misc/mclock-config.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh 2026-02-13T06:37:36.725 INFO:tasks.workunit.client.0.trial040.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T06:37:36.729 INFO:tasks.workunit.client.0.trial040.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T06:37:36.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:37:36.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/mclock-config 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:21: run: local dir=td/mclock-config 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:22: run: shift 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: export CEPH_MON=127.0.0.1:7124 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: CEPH_MON=127.0.0.1:7124 2026-02-13T06:37:36.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:25: run: export CEPH_ARGS 2026-02-13T06:37:36.731 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: uuidgen 2026-02-13T06:37:36.731 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: CEPH_ARGS+='--fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none ' 2026-02-13T06:37:36.731 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-13T06:37:36.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:28: run: CEPH_ARGS+='--debug-mclock 20 ' 2026-02-13T06:37:36.732 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: set 2026-02-13T06:37:36.732 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T06:37:36.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: local 'funcs=TEST_backfill_limit_adjustment_mclock 2026-02-13T06:37:36.733 INFO:tasks.workunit.client.0.trial040.stderr:TEST_profile_builtin_to_custom 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:TEST_profile_custom_to_builtin 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:TEST_profile_disallow_builtin_params_modify 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:TEST_profile_disallow_builtin_params_override 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:TEST_recovery_limit_adjustment_mclock' 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:36.734 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:36.735 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:36.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:36.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:36.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:36.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:36.736 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:37:36.736 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:37:36.736 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:37:36.736 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:37:36.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:37:36.737 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:37:36.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:36.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:37:36.738 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:37:36.738 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:36.738 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:37:36.739 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:37:36.739 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:37:36.741 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:37:36.741 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:36.741 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:36.741 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:37:36.742 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:37:36.742 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:37:36.742 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:37:36.743 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:37:36.743 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:36.743 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:36.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:37:36.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:37:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T06:37:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T06:37:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:37:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:37:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_backfill_limit_adjustment_mclock td/mclock-config 2026-02-13T06:37:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-13T06:37:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:238: TEST_backfill_limit_adjustment_mclock: setup td/mclock-config 2026-02-13T06:37:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:37:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:37:36.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:36.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:36.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:36.747 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:36.747 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:36.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:36.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:36.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:36.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:36.748 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:37:36.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:37:36.748 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:37:36.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:37:36.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:37:36.749 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:37:36.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:36.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:37:36.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:37:36.750 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:36.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:37:36.751 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:37:36.751 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:37:36.752 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:37:36.752 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:36.752 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:36.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:37:36.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:37:36.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:37:36.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:37:36.755 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:37:36.755 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:36.755 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:36.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:37:36.756 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:239: TEST_backfill_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T06:37:36.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T06:37:36.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:37:36.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:36.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:36.778 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:36.778 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:36.778 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:36.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:36.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:37:36.803 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:37:36.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:37:36.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:37:36.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:37:36.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:37:36.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:37:36.805 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:37:36.805 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:37:36.805 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:37:36.805 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:36.806 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:36.806 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:36.806 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:37:36.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:37:36.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get fsid 2026-02-13T06:37:36.850 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:37:36.850 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:37:36.850 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:37:36.850 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:37:36.850 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:37:36.851 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:37:36.851 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:37:36.852 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:37:36.852 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:36.852 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:36.852 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:36.852 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:37:36.852 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:37:36.852 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get mon_host 2026-02-13T06:37:36.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:240: TEST_backfill_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-13T06:37:36.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T06:37:36.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:37:36.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:37:36.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:37:36.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T06:37:36.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:37:36.994 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:37:36.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:36.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:36.995 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:36.995 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:36.995 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:36.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:36.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:37:36.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:37:37.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:242: TEST_backfill_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T06:37:37.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T06:37:37.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:37:37.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cb4824a9-5fb1-4715-aa66-1f7dd36fb1ab 2026-02-13T06:37:37.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 cb4824a9-5fb1-4715-aa66-1f7dd36fb1ab' 2026-02-13T06:37:37.020 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 cb4824a9-5fb1-4715-aa66-1f7dd36fb1ab 2026-02-13T06:37:37.020 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:37:37.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCxxo5pyneGARAAmy41P8+H7rKKN41Fb3gd8w== 2026-02-13T06:37:37.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCxxo5pyneGARAAmy41P8+H7rKKN41Fb3gd8w=="}' 2026-02-13T06:37:37.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cb4824a9-5fb1-4715-aa66-1f7dd36fb1ab -i td/mclock-config/0/new.json 2026-02-13T06:37:37.128 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:37:37.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T06:37:37.135 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQCxxo5pyneGARAAmy41P8+H7rKKN41Fb3gd8w== --osd-uuid cb4824a9-5fb1-4715-aa66-1f7dd36fb1ab 2026-02-13T06:37:37.153 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:37.150+0000 7fa96de0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:37.154 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:37.152+0000 7fa96de0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:37.155 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:37.153+0000 7fa96de0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:37.156 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:37.153+0000 7fa96de0e900 -1 bdev(0x55bec6772c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:37:37.156 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:37.153+0000 7fa96de0e900 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T06:37:37.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T06:37:37.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:37:37.552 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:37:37.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:37:37.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:37:37.656 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:37:37.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:37:37.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T06:37:37.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:37:37.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:37:37.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:37:37.672 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:37.670+0000 7fa3b5d93900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:37.674 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:37.671+0000 7fa3b5d93900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:37.675 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:37.672+0000 7fa3b5d93900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:37.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:37:37.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:37:37.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:37:37.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:37:37.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:37:37.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:37.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:37:37.757 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:37:37.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:37.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:37:37.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:37:37.863 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:37.860+0000 7fa3b5d93900 -1 Falling back to public interface 2026-02-13T06:37:38.261 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:38.259+0000 7fa3b5d93900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:37:38.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:37:38.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:38.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:37:38.854 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:37:38.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:38.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:37:38.951 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3071577861,v1:127.0.0.1:6801/3071577861] [v2:127.0.0.1:6802/3071577861,v1:127.0.0.1:6803/3071577861] exists,up cb4824a9-5fb1-4715-aa66-1f7dd36fb1ab 2026-02-13T06:37:38.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:37:38.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:37:38.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:37:38.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-13T06:37:38.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:38.953 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:38.953 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:38.953 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:38.953 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:38.953 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:38.954 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:38.954 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:38.954 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:38.954 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T06:37:39.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: local backfills=1 2026-02-13T06:37:39.005 INFO:tasks.workunit.client.0.trial040.stdout:osd_max_backfills: 1 2026-02-13T06:37:39.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: echo 'osd_max_backfills: 1' 2026-02-13T06:37:39.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:251: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-13T06:37:39.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:252: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-13T06:37:41.109 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-13T06:37:41.109 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:41.110 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:41.111 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:41.111 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:41.111 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:41.111 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:41.111 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:41.111 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:41.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:41.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T06:37:41.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: local max_backfills=1 2026-02-13T06:37:41.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-13T06:37:41.160 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-13T06:37:41.161 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:41.161 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:41.161 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:41.161 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:41.162 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:41.162 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:41.162 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:41.162 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:41.162 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:41.162 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T06:37:41.208 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-13T06:37:41.209 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:260: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-13T06:37:41.210 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-13T06:37:41.210 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:41.210 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:41.210 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:41.210 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:41.210 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:41.210 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:41.210 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:41.211 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:41.211 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:41.211 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T06:37:41.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-13T06:37:41.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:263: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-13T06:37:41.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:267: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-13T06:37:41.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:268: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-13T06:37:41.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:269: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-13T06:37:43.717 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-13T06:37:43.717 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:43.719 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:43.719 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:43.719 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:43.719 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:43.719 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:43.719 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:43.719 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:43.719 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:43.719 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T06:37:43.769 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T06:37:43.769 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:272: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T06:37:43.770 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-13T06:37:43.770 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:43.771 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:43.771 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:43.771 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:43.771 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:43.771 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:43.771 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:43.772 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:43.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:43.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T06:37:43.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T06:37:43.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:277: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T06:37:43.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-13T06:37:43.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:43.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:43.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:43.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:43.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:43.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:43.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:43.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:43.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:43.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T06:37:43.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T06:37:43.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:280: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T06:37:43.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:284: TEST_backfill_limit_adjustment_mclock: kill_daemons td/mclock-config TERM osd 2026-02-13T06:37:43.870 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:43.870 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:43.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:43.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:43.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:43.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:43.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:285: TEST_backfill_limit_adjustment_mclock: ceph osd down 0 2026-02-13T06:37:44.198 INFO:tasks.workunit.client.0.trial040.stderr:osd.0 is already down. 2026-02-13T06:37:44.205 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:286: TEST_backfill_limit_adjustment_mclock: wait_for_osd down 0 2026-02-13T06:37:44.205 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-13T06:37:44.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:37:44.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:37:44.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:37:44.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:44.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:37:44.206 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:37:44.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:44.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-13T06:37:44.436 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/3071577861,v1:127.0.0.1:6801/3071577861] [v2:127.0.0.1:6802/3071577861,v1:127.0.0.1:6803/3071577861] exists cb4824a9-5fb1-4715-aa66-1f7dd36fb1ab 2026-02-13T06:37:44.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:287: TEST_backfill_limit_adjustment_mclock: activate_osd td/mclock-config 0 --osd-op-queue=mclock_scheduler 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/mclock-config 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/mclock-config/0 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:37:44.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:44.441 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=--osd-op-queue=mclock_scheduler 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/mclock-config/0 2026-02-13T06:37:44.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T06:37:44.443 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:37:44.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler 2026-02-13T06:37:44.444 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/mclock-config/0/whoami 2026-02-13T06:37:44.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T06:37:44.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T06:37:44.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T06:37:44.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T06:37:44.455 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:44.452+0000 7fdcc9f1c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:44.456 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:44.454+0000 7fdcc9f1c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:44.457 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:44.455+0000 7fdcc9f1c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:44.587 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:44.585+0000 7fdcc9f1c900 -1 Falling back to public interface 2026-02-13T06:37:44.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T06:37:44.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:37:44.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:37:44.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:37:44.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:37:44.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:44.664 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:37:44.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:37:44.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:44.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:37:44.886 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:44.884+0000 7fdcc9f1c900 -1 osd.0 6 log_to_monitors true 2026-02-13T06:37:44.891 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:37:45.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:37:45.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:45.893 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:37:45.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:37:45.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:45.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:37:46.119 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 6 last_clean_interval [5,6) [v2:127.0.0.1:6800/1057409782,v1:127.0.0.1:6801/1057409782] [v2:127.0.0.1:6802/1057409782,v1:127.0.0.1:6803/1057409782] exists,up cb4824a9-5fb1-4715-aa66-1f7dd36fb1ab 2026-02-13T06:37:46.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:37:46.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:37:46.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:37:46.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-13T06:37:46.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:46.121 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:46.122 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:46.122 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:46.122 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:46.122 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.122 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.122 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:46.122 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:46.122 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T06:37:46.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T06:37:46.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:291: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T06:37:46.171 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-13T06:37:46.171 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:46.172 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:46.172 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:46.172 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:46.172 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:46.172 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.172 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.172 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:46.172 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:46.172 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T06:37:46.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T06:37:46.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:296: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T06:37:46.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-13T06:37:46.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T06:37:46.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:37:46.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:46.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:46.222 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:46.222 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.222 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.223 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:46.223 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:37:46.223 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T06:37:46.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T06:37:46.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:299: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T06:37:46.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:301: TEST_backfill_limit_adjustment_mclock: teardown td/mclock-config 2026-02-13T06:37:46.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:46.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:46.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:46.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:46.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:46.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:46.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:46.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:46.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:46.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:37:46.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:37:46.379 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:37:46.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:37:46.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:37:46.381 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:37:46.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:46.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:37:46.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:37:46.383 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:46.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:37:46.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:37:46.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:37:46.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:37:46.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:37:46.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:37:46.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:37:46.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T06:37:46.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:46.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:46.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:46.391 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:46.391 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:46.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:46.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:46.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:46.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:46.392 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:37:46.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:37:46.393 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:37:46.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:37:46.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:37:46.394 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:37:46.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:46.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:37:46.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:37:46.395 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:46.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:37:46.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:37:46.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:37:46.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:37:46.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:37:46.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:37:46.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:37:46.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T06:37:46.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T06:37:46.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:37:46.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:37:46.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:46.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:46.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:46.401 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:46.401 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:46.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:46.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:46.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:46.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:46.402 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:37:46.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:37:46.402 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:37:46.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:37:46.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:37:46.404 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:37:46.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:46.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:37:46.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:37:46.405 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:46.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:37:46.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:37:46.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:37:46.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:37:46.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:37:46.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:37:46.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:37:46.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:37:46.410 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:37:46.410 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.410 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:37:46.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:37:46.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:37:46.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:37:46.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:37:46.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_builtin_to_custom td/mclock-config 2026-02-13T06:37:46.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:39: TEST_profile_builtin_to_custom: local dir=td/mclock-config 2026-02-13T06:37:46.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:41: TEST_profile_builtin_to_custom: run_mon td/mclock-config a 2026-02-13T06:37:46.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T06:37:46.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:37:46.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:37:46.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:37:46.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T06:37:46.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T06:37:46.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:37:46.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:46.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:46.433 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:46.433 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.433 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.433 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:46.433 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:37:46.460 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:37:46.461 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.462 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:37:46.463 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:37:46.463 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get fsid 2026-02-13T06:37:46.507 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:37:46.507 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:37:46.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:37:46.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:37:46.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:37:46.509 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:37:46.509 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:37:46.509 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:37:46.509 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:46.509 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.509 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.509 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:37:46.509 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:37:46.510 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get mon_host 2026-02-13T06:37:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:42: TEST_profile_builtin_to_custom: run_mgr td/mclock-config x 2026-02-13T06:37:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T06:37:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:37:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:37:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:37:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T06:37:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:37:46.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:37:46.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:46.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:46.658 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:46.658 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.658 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:46.659 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:37:46.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:37:46.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:43: TEST_profile_builtin_to_custom: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T06:37:46.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T06:37:46.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:37:46.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T06:37:46.679 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T06:37:46.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:37:46.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:37:46.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:37:46.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:37:46.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:37:46.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T06:37:46.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T06:37:46.681 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:37:46.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1d64c08b-15aa-493b-9f9e-f14c0f59b90c 2026-02-13T06:37:46.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1d64c08b-15aa-493b-9f9e-f14c0f59b90c' 2026-02-13T06:37:46.682 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 1d64c08b-15aa-493b-9f9e-f14c0f59b90c 2026-02-13T06:37:46.682 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:37:46.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC6xo5pMjX+KBAA0sgFIw+o2DUmi4ezPk2QEw== 2026-02-13T06:37:46.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC6xo5pMjX+KBAA0sgFIw+o2DUmi4ezPk2QEw=="}' 2026-02-13T06:37:46.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1d64c08b-15aa-493b-9f9e-f14c0f59b90c -i td/mclock-config/0/new.json 2026-02-13T06:37:46.791 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:37:46.797 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T06:37:46.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQC6xo5pMjX+KBAA0sgFIw+o2DUmi4ezPk2QEw== --osd-uuid 1d64c08b-15aa-493b-9f9e-f14c0f59b90c 2026-02-13T06:37:46.816 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:46.814+0000 7f73b92ff900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:46.818 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:46.816+0000 7f73b92ff900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:46.819 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:46.817+0000 7f73b92ff900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:46.819 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:46.817+0000 7f73b92ff900 -1 bdev(0x55a80f684c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:37:46.819 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:46.817+0000 7f73b92ff900 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T06:37:47.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T06:37:47.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:37:47.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:37:47.290 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:37:47.290 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:37:47.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:37:47.396 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:37:47.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T06:37:47.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:37:47.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:37:47.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:37:47.411 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:47.410+0000 7fd4001c7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:47.413 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:47.412+0000 7fd4001c7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:47.414 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:47.413+0000 7fd4001c7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:47.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:37:47.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:37:47.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:37:47.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:37:47.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:37:47.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:47.496 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:37:47.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:37:47.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:47.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:37:47.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:37:47.660 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:47.659+0000 7fd4001c7900 -1 Falling back to public interface 2026-02-13T06:37:48.093 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:48.091+0000 7fd4001c7900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:37:48.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:37:48.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:48.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:37:48.595 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:37:48.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:48.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:37:48.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:37:49.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:37:49.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:49.698 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-13T06:37:49.698 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T06:37:49.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:49.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:37:49.919 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/87205935,v1:127.0.0.1:6801/87205935] [v2:127.0.0.1:6802/87205935,v1:127.0.0.1:6803/87205935] exists,up 1d64c08b-15aa-493b-9f9e-f14c0f59b90c 2026-02-13T06:37:49.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:37:49.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:37:49.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:37:49.921 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_profile 2026-02-13T06:37:50.145 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: local mclock_profile=balanced 2026-02-13T06:37:50.145 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:47: TEST_profile_builtin_to_custom: test balanced = balanced 2026-02-13T06:37:50.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-13T06:37:50.146 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-13T06:37:50.146 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:50.147 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:50.147 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:50.147 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:50.147 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:50.147 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:50.147 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-13T06:37:50.147 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T06:37:50.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: mclock_profile='"high_recovery_ops"' 2026-02-13T06:37:50.195 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: eval echo '"high_recovery_ops"' 2026-02-13T06:37:50.195 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: echo high_recovery_ops 2026-02-13T06:37:50.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: mclock_profile=high_recovery_ops 2026-02-13T06:37:50.196 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:54: TEST_profile_builtin_to_custom: test high_recovery_ops = high_recovery_ops 2026-02-13T06:37:50.196 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:57: TEST_profile_builtin_to_custom: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-13T06:37:50.252 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:37:50.253 INFO:tasks.workunit.client.0.trial040.stdout: "success": "" 2026-02-13T06:37:50.253 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:37:50.258 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-13T06:37:50.258 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-13T06:37:50.258 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:50.258 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:50.258 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:50.258 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:50.258 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:50.258 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:50.259 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-13T06:37:50.259 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T06:37:50.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: mclock_profile='"custom"' 2026-02-13T06:37:50.309 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: eval echo '"custom"' 2026-02-13T06:37:50.309 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: echo custom 2026-02-13T06:37:50.309 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: mclock_profile=custom 2026-02-13T06:37:50.309 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: test custom = custom 2026-02-13T06:37:50.309 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-13T06:37:50.309 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: bc 2026-02-13T06:37:50.309 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-13T06:37:50.309 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:50.309 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:50.310 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:50.310 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:50.310 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:50.310 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:50.310 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-13T06:37:50.310 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:37:50.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: local client_res=0.300000 2026-02-13T06:37:50.360 INFO:tasks.workunit.client.0.trial040.stdout:client_res = 0.300000 2026-02-13T06:37:50.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:69: TEST_profile_builtin_to_custom: echo 'client_res = 0.300000' 2026-02-13T06:37:50.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: echo '0.300000 + 0.1' 2026-02-13T06:37:50.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: bc -l 2026-02-13T06:37:50.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: local client_res_new=.400000 2026-02-13T06:37:50.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:71: TEST_profile_builtin_to_custom: echo 'client_res_new = .400000' 2026-02-13T06:37:50.361 INFO:tasks.workunit.client.0.trial040.stdout:client_res_new = .400000 2026-02-13T06:37:50.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:72: TEST_profile_builtin_to_custom: ceph config set osd.0 osd_mclock_scheduler_client_res .400000 2026-02-13T06:37:50.588 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T06:37:50.823 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: local res=0.400000 2026-02-13T06:37:50.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: echo '0.400000 != .400000' 2026-02-13T06:37:50.824 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: bc -l 2026-02-13T06:37:50.824 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-13T06:37:50.826 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-13T06:37:50.826 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: bc 2026-02-13T06:37:50.826 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-13T06:37:50.826 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:50.827 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:50.827 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:50.827 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:50.827 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:50.827 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:50.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-13T06:37:50.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:37:50.874 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: res=0.400000 2026-02-13T06:37:50.874 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: echo '0.400000 != .400000' 2026-02-13T06:37:50.874 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: bc -l 2026-02-13T06:37:50.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-13T06:37:50.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:89: TEST_profile_builtin_to_custom: teardown td/mclock-config 2026-02-13T06:37:50.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:50.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:50.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:50.877 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:50.877 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:50.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:50.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:50.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:50.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:50.983 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:37:50.983 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:37:50.984 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:37:50.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:37:50.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:37:50.986 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:37:50.986 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:50.987 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:37:50.987 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:37:50.987 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:50.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:37:50.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:37:50.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:37:50.992 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:37:50.992 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:50.992 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:50.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:37:50.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:37:50.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:37:50.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T06:37:50.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:50.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:50.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:50.994 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:50.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:50.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:50.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:50.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:50.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:50.996 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:37:50.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:37:50.997 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:37:50.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:37:50.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:37:50.998 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:37:50.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:50.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:37:50.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:37:50.999 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:51.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:37:51.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:37:51.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:37:51.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:37:51.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:37:51.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:37:51.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:37:51.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T06:37:51.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T06:37:51.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:37:51.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:37:51.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:51.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:51.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:51.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:51.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:51.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:51.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:51.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:51.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:51.006 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:37:51.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:37:51.006 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:37:51.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:37:51.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:37:51.008 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:37:51.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:51.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:37:51.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:37:51.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:51.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:37:51.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:37:51.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:37:51.011 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:37:51.012 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.012 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:37:51.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:37:51.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:37:51.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:37:51.014 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:37:51.014 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.014 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:37:51.015 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:37:51.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_custom_to_builtin td/mclock-config 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:93: TEST_profile_custom_to_builtin: local dir=td/mclock-config 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:95: TEST_profile_custom_to_builtin: setup td/mclock-config 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:51.017 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:51.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:51.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:51.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:51.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:37:51.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:37:51.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:37:51.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:37:51.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:37:51.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:37:51.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:37:51.020 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:37:51.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:51.021 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:37:51.021 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:37:51.021 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:37:51.022 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:37:51.022 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:37:51.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:37:51.024 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:37:51.024 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.024 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:37:51.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:37:51.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:37:51.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:37:51.026 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:37:51.026 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.026 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:37:51.028 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:37:51.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:37:51.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:37:51.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:37:51.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:96: TEST_profile_custom_to_builtin: run_mon td/mclock-config a 2026-02-13T06:37:51.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T06:37:51.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:37:51.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:37:51.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:37:51.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T06:37:51.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T06:37:51.047 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:37:51.047 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:51.047 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:51.047 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:51.048 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.048 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:51.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:37:51.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:37:51.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:37:51.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:37:51.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:37:51.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:37:51.074 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:37:51.074 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:37:51.074 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:37:51.074 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:37:51.075 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:51.075 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.075 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.075 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:37:51.075 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:37:51.075 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get fsid 2026-02-13T06:37:51.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:37:51.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:37:51.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:37:51.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:37:51.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:37:51.119 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:37:51.119 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:37:51.119 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:37:51.120 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:51.120 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.120 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.120 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:37:51.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:37:51.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get mon_host 2026-02-13T06:37:51.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:97: TEST_profile_custom_to_builtin: run_mgr td/mclock-config x 2026-02-13T06:37:51.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T06:37:51.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:37:51.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:37:51.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:37:51.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T06:37:51.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:37:51.264 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:37:51.264 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:51.264 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:51.264 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:51.264 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.264 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.264 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:51.265 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:37:51.265 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:37:51.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:98: TEST_profile_custom_to_builtin: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T06:37:51.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T06:37:51.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:37:51.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:37:51.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:37:51.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T06:37:51.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T06:37:51.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:37:51.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:51.287 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:37:51.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T06:37:51.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T06:37:51.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:37:51.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c1abdc67-4f1e-4e5f-8d44-f9ff9171387f 2026-02-13T06:37:51.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c1abdc67-4f1e-4e5f-8d44-f9ff9171387f' 2026-02-13T06:37:51.289 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 c1abdc67-4f1e-4e5f-8d44-f9ff9171387f 2026-02-13T06:37:51.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:37:51.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC/xo5ppHSPERAAPF4jvrpPZF1zH8RqfG109A== 2026-02-13T06:37:51.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC/xo5ppHSPERAAPF4jvrpPZF1zH8RqfG109A=="}' 2026-02-13T06:37:51.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c1abdc67-4f1e-4e5f-8d44-f9ff9171387f -i td/mclock-config/0/new.json 2026-02-13T06:37:51.399 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:37:51.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T06:37:51.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQC/xo5ppHSPERAAPF4jvrpPZF1zH8RqfG109A== --osd-uuid c1abdc67-4f1e-4e5f-8d44-f9ff9171387f 2026-02-13T06:37:51.423 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:51.422+0000 7f65c99e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:51.425 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:51.423+0000 7f65c99e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:51.426 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:51.424+0000 7f65c99e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:51.426 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:51.424+0000 7f65c99e0900 -1 bdev(0x55809e3cac00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:37:51.426 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:51.425+0000 7f65c99e0900 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T06:37:51.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T06:37:51.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:37:51.858 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:37:51.858 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:37:51.858 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:37:51.963 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:37:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:37:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T06:37:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:37:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:37:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:37:51.980 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:51.979+0000 7fb1b69c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:51.982 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:51.980+0000 7fb1b69c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:51.983 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:51.981+0000 7fb1b69c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:37:52.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:37:52.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:37:52.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:37:52.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:37:52.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:37:52.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:52.064 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:37:52.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:37:52.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:52.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:37:52.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:37:52.177 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:52.176+0000 7fb1b69c6900 -1 Falling back to public interface 2026-02-13T06:37:52.529 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:37:52.527+0000 7fb1b69c6900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:37:53.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:37:53.166 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:37:53.166 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:37:53.166 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:37:53.167 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:37:53.167 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:37:53.263 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2623999446,v1:127.0.0.1:6801/2623999446] [v2:127.0.0.1:6802/2623999446,v1:127.0.0.1:6803/2623999446] exists,up c1abdc67-4f1e-4e5f-8d44-f9ff9171387f 2026-02-13T06:37:53.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:37:53.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:37:53.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:37:53.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:101: TEST_profile_custom_to_builtin: local def_mclock_profile 2026-02-13T06:37:53.264 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_profile 2026-02-13T06:37:53.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: def_mclock_profile=balanced 2026-02-13T06:37:53.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:103: TEST_profile_custom_to_builtin: test balanced = balanced 2026-02-13T06:37:53.367 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-13T06:37:53.367 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T06:37:53.368 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:53.368 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:53.368 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:53.368 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:53.368 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:53.368 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:53.368 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T06:37:53.368 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T06:37:53.416 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: local 'orig_mclock_profile="high_recovery_ops"' 2026-02-13T06:37:53.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: eval echo '"high_recovery_ops"' 2026-02-13T06:37:53.417 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: echo high_recovery_ops 2026-02-13T06:37:53.418 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: orig_mclock_profile=high_recovery_ops 2026-02-13T06:37:53.418 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:110: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-13T06:37:53.418 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:113: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-13T06:37:53.476 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:37:53.476 INFO:tasks.workunit.client.0.trial040.stdout: "success": "" 2026-02-13T06:37:53.476 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:37:53.481 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-13T06:37:53.482 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T06:37:53.482 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:53.482 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:53.482 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:53.482 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:53.482 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:53.482 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:53.483 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T06:37:53.483 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T06:37:53.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: local 'mclock_profile="custom"' 2026-02-13T06:37:53.531 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: eval echo '"custom"' 2026-02-13T06:37:53.531 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: echo custom 2026-02-13T06:37:53.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: mclock_profile=custom 2026-02-13T06:37:53.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:120: TEST_profile_custom_to_builtin: test custom = custom 2026-02-13T06:37:53.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: local client_res 2026-02-13T06:37:53.532 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: bc 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T06:37:53.533 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:37:53.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: client_res=0.300000 2026-02-13T06:37:53.582 INFO:tasks.workunit.client.0.trial040.stdout:Original client_res for osd.0 = 0.300000 2026-02-13T06:37:53.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: echo 'Original client_res for osd.0 = 0.300000' 2026-02-13T06:37:53.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: echo '0.300000 + 0.1' 2026-02-13T06:37:53.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: bc -l 2026-02-13T06:37:53.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: local client_res_new=.400000 2026-02-13T06:37:53.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:131: TEST_profile_custom_to_builtin: echo 'client_res_new = .400000' 2026-02-13T06:37:53.584 INFO:tasks.workunit.client.0.trial040.stdout:client_res_new = .400000 2026-02-13T06:37:53.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:132: TEST_profile_custom_to_builtin: ceph config set osd osd_mclock_scheduler_client_res .400000 2026-02-13T06:37:53.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T06:37:54.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: local res=0.400000 2026-02-13T06:37:54.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: echo '0.400000 != .400000' 2026-02-13T06:37:54.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: bc -l 2026-02-13T06:37:54.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T06:37:54.041 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-13T06:37:54.041 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: bc 2026-02-13T06:37:54.041 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T06:37:54.041 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:54.041 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:54.041 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:54.041 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:54.041 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:54.041 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:54.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T06:37:54.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:37:54.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: res=0.400000 2026-02-13T06:37:54.089 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: echo '0.400000 != .400000' 2026-02-13T06:37:54.089 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: bc -l 2026-02-13T06:37:54.090 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T06:37:54.090 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:154: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile high_recovery_ops 2026-02-13T06:37:54.148 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:37:54.148 INFO:tasks.workunit.client.0.trial040.stdout: "success": "" 2026-02-13T06:37:54.148 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:37:54.154 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-13T06:37:54.155 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T06:37:54.155 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:54.155 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:54.156 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:54.156 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:54.156 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:54.156 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:54.156 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T06:37:54.156 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T06:37:54.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: eval 'mclock_profile="high_recovery_ops"' 2026-02-13T06:37:54.204 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: mclock_profile=high_recovery_ops 2026-02-13T06:37:54.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:160: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-13T06:37:54.204 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T06:37:54.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: local res=0.400000 2026-02-13T06:37:54.429 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: echo '0.400000 != .400000' 2026-02-13T06:37:54.429 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: bc -l 2026-02-13T06:37:54.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T06:37:54.431 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-13T06:37:54.431 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: bc 2026-02-13T06:37:54.431 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T06:37:54.431 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:54.432 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:54.432 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:54.432 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:54.432 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:54.432 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:54.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T06:37:54.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:37:54.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: res=0.400000 2026-02-13T06:37:54.480 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: echo '0.400000 != .400000' 2026-02-13T06:37:54.480 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: bc -l 2026-02-13T06:37:54.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T06:37:54.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:178: TEST_profile_custom_to_builtin: ceph config rm osd osd_mclock_scheduler_client_res 2026-02-13T06:37:54.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:180: TEST_profile_custom_to_builtin: sleep 5 2026-02-13T06:37:59.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T06:37:59.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: res=0.000000 2026-02-13T06:37:59.929 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: echo '0.000000 != 0.0' 2026-02-13T06:37:59.929 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: bc -l 2026-02-13T06:37:59.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T06:37:59.931 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-13T06:37:59.931 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T06:37:59.931 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: bc 2026-02-13T06:37:59.931 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:37:59.931 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:37:59.931 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:37:59.932 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:37:59.932 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:37:59.932 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:37:59.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T06:37:59.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:37:59.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: res=0.300000 2026-02-13T06:37:59.980 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: echo '0.300000 != 0.300000' 2026-02-13T06:37:59.980 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: bc -l 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:198: TEST_profile_custom_to_builtin: teardown td/mclock-config 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:37:59.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:00.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:00.089 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:00.090 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:00.090 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:00.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:00.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:00.092 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:00.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:00.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:00.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:00.094 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:00.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:00.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:00.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:00.099 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:00.099 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.099 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.099 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:00.100 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:00.101 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:00.102 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:00.102 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:00.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:00.103 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:00.104 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:00.104 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:00.104 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:00.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:00.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:00.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:00.106 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:00.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:00.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:00.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:00.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:00.109 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.109 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:00.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:00.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:00.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:00.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:00.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:00.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:00.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:00.113 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:00.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:00.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:00.114 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:00.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:00.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:00.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:00.116 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:00.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:00.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:00.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:00.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:00.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.119 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:00.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:00.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:00.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:38:00.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:38:00.121 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.121 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.121 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:38:00.122 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:38:00.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_disallow_builtin_params_modify td/mclock-config 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:305: TEST_profile_disallow_builtin_params_modify: local dir=td/mclock-config 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:307: TEST_profile_disallow_builtin_params_modify: setup td/mclock-config 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:00.124 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:00.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:00.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:00.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:00.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:00.126 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:00.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:00.126 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:00.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:00.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:00.127 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:00.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:00.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:00.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:00.128 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:00.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:00.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:00.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:00.131 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:00.131 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.131 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.131 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:00.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:00.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:00.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:38:00.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:38:00.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.133 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:38:00.134 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:38:00.135 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:38:00.135 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:38:00.135 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:38:00.135 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:308: TEST_profile_disallow_builtin_params_modify: run_mon td/mclock-config a 2026-02-13T06:38:00.136 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T06:38:00.136 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:38:00.136 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:38:00.136 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:38:00.136 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T06:38:00.136 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T06:38:00.155 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:38:00.155 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:00.155 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:00.155 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:00.155 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.155 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.155 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:00.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:38:00.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:38:00.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:38:00.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:38:00.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:38:00.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:38:00.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:38:00.183 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:38:00.183 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:38:00.183 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:38:00.183 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:00.184 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.184 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.184 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:38:00.184 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:38:00.184 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get fsid 2026-02-13T06:38:00.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:38:00.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:38:00.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:38:00.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:38:00.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:38:00.229 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:38:00.230 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:38:00.230 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:38:00.230 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:00.230 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.230 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.230 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:38:00.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:38:00.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get mon_host 2026-02-13T06:38:00.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:309: TEST_profile_disallow_builtin_params_modify: run_mgr td/mclock-config x 2026-02-13T06:38:00.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T06:38:00.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:38:00.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:38:00.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:38:00.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T06:38:00.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:38:00.375 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:38:00.375 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:00.375 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:00.375 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:00.375 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.375 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.375 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:00.376 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:38:00.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:38:00.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:311: TEST_profile_disallow_builtin_params_modify: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T06:38:00.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:00.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:38:00.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=afe6dad0-ab7f-460d-be78-482f7e08cd60 2026-02-13T06:38:00.400 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 afe6dad0-ab7f-460d-be78-482f7e08cd60 2026-02-13T06:38:00.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 afe6dad0-ab7f-460d-be78-482f7e08cd60' 2026-02-13T06:38:00.400 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:38:00.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDIxo5pVCU/GBAA9Ew3+EMITiyaAKvHg1Mdxg== 2026-02-13T06:38:00.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDIxo5pVCU/GBAA9Ew3+EMITiyaAKvHg1Mdxg=="}' 2026-02-13T06:38:00.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new afe6dad0-ab7f-460d-be78-482f7e08cd60 -i td/mclock-config/0/new.json 2026-02-13T06:38:00.511 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:00.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T06:38:00.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQDIxo5pVCU/GBAA9Ew3+EMITiyaAKvHg1Mdxg== --osd-uuid afe6dad0-ab7f-460d-be78-482f7e08cd60 2026-02-13T06:38:00.536 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:00.535+0000 7f8e664f6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:00.538 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:00.537+0000 7f8e664f6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:00.539 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:00.538+0000 7f8e664f6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:00.540 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:00.538+0000 7f8e664f6900 -1 bdev(0x55d9fddd4c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:38:00.540 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:00.538+0000 7f8e664f6900 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T06:38:00.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T06:38:00.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:38:00.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:38:00.980 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:38:00.980 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:38:01.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:38:01.086 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:38:01.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T06:38:01.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:38:01.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:38:01.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:38:01.102 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:01.100+0000 7f644172d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:01.103 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:01.102+0000 7f644172d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:01.104 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:01.103+0000 7f644172d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:01.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:38:01.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:38:01.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:38:01.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:38:01.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:38:01.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:01.186 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:01.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:38:01.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:01.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:38:01.283 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:38:01.341 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:01.339+0000 7f644172d900 -1 Falling back to public interface 2026-02-13T06:38:01.644 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:01.643+0000 7f644172d900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:38:02.285 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:38:02.285 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:02.285 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:38:02.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:38:02.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:02.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:38:02.384 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1271348888,v1:127.0.0.1:6801/1271348888] [v2:127.0.0.1:6802/1271348888,v1:127.0.0.1:6803/1271348888] exists,up afe6dad0-ab7f-460d-be78-482f7e08cd60 2026-02-13T06:38:02.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:38:02.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:38:02.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:38:02.385 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:314: TEST_profile_disallow_builtin_params_modify: ceph config get osd.0 osd_mclock_profile 2026-02-13T06:38:02.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:314: TEST_profile_disallow_builtin_params_modify: local def_mclock_profile=balanced 2026-02-13T06:38:02.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:315: TEST_profile_disallow_builtin_params_modify: test balanced = balanced 2026-02-13T06:38:02.485 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:320: TEST_profile_disallow_builtin_params_modify: jq .osd_mclock_profile 2026-02-13T06:38:02.485 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:320: TEST_profile_disallow_builtin_params_modify: get_asok_path osd.0 2026-02-13T06:38:02.486 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:02.486 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:02.486 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:02.486 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:02.486 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:02.486 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:02.486 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:320: TEST_profile_disallow_builtin_params_modify: CEPH_ARGS= 2026-02-13T06:38:02.486 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:320: TEST_profile_disallow_builtin_params_modify: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T06:38:02.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:320: TEST_profile_disallow_builtin_params_modify: local 'cur_mclock_profile="high_recovery_ops"' 2026-02-13T06:38:02.535 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:321: TEST_profile_disallow_builtin_params_modify: eval echo '"high_recovery_ops"' 2026-02-13T06:38:02.535 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:321: TEST_profile_disallow_builtin_params_modify: echo high_recovery_ops 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:321: TEST_profile_disallow_builtin_params_modify: cur_mclock_profile=high_recovery_ops 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:322: TEST_profile_disallow_builtin_params_modify: test high_recovery_ops = high_recovery_ops 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:324: TEST_profile_disallow_builtin_params_modify: options=('osd_mclock_scheduler_background_recovery_res' 'osd_mclock_scheduler_client_res') 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:324: TEST_profile_disallow_builtin_params_modify: declare -a options 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:327: TEST_profile_disallow_builtin_params_modify: local retries=10 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:328: TEST_profile_disallow_builtin_params_modify: local errors=0 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:329: TEST_profile_disallow_builtin_params_modify: for opt in "${options[@]}" 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: jq .osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: bc 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: get_asok_path osd.0 2026-02-13T06:38:02.536 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:02.537 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:02.537 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:02.537 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:02.537 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:02.537 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:02.537 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: CEPH_ARGS= 2026-02-13T06:38:02.537 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:02.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: local opt_val_orig=0.700000 2026-02-13T06:38:02.587 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:334: TEST_profile_disallow_builtin_params_modify: echo '0.700000 + 0.1' 2026-02-13T06:38:02.587 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:334: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:02.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:334: TEST_profile_disallow_builtin_params_modify: local opt_val_new=.800000 2026-02-13T06:38:02.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:335: TEST_profile_disallow_builtin_params_modify: ceph config set osd.0 osd_mclock_scheduler_background_recovery_res .800000 2026-02-13T06:38:02.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:338: TEST_profile_disallow_builtin_params_modify: expr 10 - 1 2026-02-13T06:38:02.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:338: TEST_profile_disallow_builtin_params_modify: seq 0 9 2026-02-13T06:38:02.824 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:338: TEST_profile_disallow_builtin_params_modify: for count in $(seq 0 $(expr $retries - 1)) 2026-02-13T06:38:02.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:340: TEST_profile_disallow_builtin_params_modify: errors=0 2026-02-13T06:38:02.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:341: TEST_profile_disallow_builtin_params_modify: sleep 2 2026-02-13T06:38:04.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:343: TEST_profile_disallow_builtin_params_modify: echo 'Check configuration values - Attempt#: 0' 2026-02-13T06:38:04.826 INFO:tasks.workunit.client.0.trial040.stdout:Check configuration values - Attempt#: 0 2026-02-13T06:38:04.826 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:345: TEST_profile_disallow_builtin_params_modify: ceph config get osd.0 osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:05.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:345: TEST_profile_disallow_builtin_params_modify: local res=0.000000 2026-02-13T06:38:05.051 INFO:tasks.workunit.client.0.trial040.stdout:Mon db (or default): osd.0 osd_mclock_scheduler_background_recovery_res = 0.000000 2026-02-13T06:38:05.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:346: TEST_profile_disallow_builtin_params_modify: echo 'Mon db (or default): osd.0 osd_mclock_scheduler_background_recovery_res = 0.000000' 2026-02-13T06:38:05.052 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:347: TEST_profile_disallow_builtin_params_modify: echo '0.000000 == .800000' 2026-02-13T06:38:05.052 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:347: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:05.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:347: TEST_profile_disallow_builtin_params_modify: (( 0 )) 2026-02-13T06:38:05.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: ceph config show osd.0 2026-02-13T06:38:05.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: grep osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:05.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: awk '{ print $2 }' 2026-02-13T06:38:05.054 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: bc 2026-02-13T06:38:05.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: res=.700000 2026-02-13T06:38:05.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:354: TEST_profile_disallow_builtin_params_modify: echo 'Running config: osd.0 osd_mclock_scheduler_background_recovery_res = .700000' 2026-02-13T06:38:05.278 INFO:tasks.workunit.client.0.trial040.stdout:Running config: osd.0 osd_mclock_scheduler_background_recovery_res = .700000 2026-02-13T06:38:05.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: echo '.700000 == .800000' 2026-02-13T06:38:05.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:05.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: echo '.700000 != 0.700000' 2026-02-13T06:38:05.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:05.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: (( 0 || 0 )) 2026-02-13T06:38:05.283 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: jq .osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:05.283 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: bc 2026-02-13T06:38:05.283 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: get_asok_path osd.0 2026-02-13T06:38:05.283 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:05.283 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:05.283 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:05.284 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:05.284 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:05.284 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:05.284 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: CEPH_ARGS= 2026-02-13T06:38:05.284 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:05.331 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: res=0.700000 2026-02-13T06:38:05.331 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:363: TEST_profile_disallow_builtin_params_modify: echo 'Values map: osd.0 osd_mclock_scheduler_background_recovery_res = 0.700000' 2026-02-13T06:38:05.332 INFO:tasks.workunit.client.0.trial040.stdout:Values map: osd.0 osd_mclock_scheduler_background_recovery_res = 0.700000 2026-02-13T06:38:05.332 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: echo '0.700000 == .800000' 2026-02-13T06:38:05.332 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:05.333 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: echo '0.700000 != 0.700000' 2026-02-13T06:38:05.333 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:05.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: (( 0 || 0 )) 2026-02-13T06:38:05.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:370: TEST_profile_disallow_builtin_params_modify: '[' 0 -eq 0 ']' 2026-02-13T06:38:05.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:372: TEST_profile_disallow_builtin_params_modify: break 2026-02-13T06:38:05.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:329: TEST_profile_disallow_builtin_params_modify: for opt in "${options[@]}" 2026-02-13T06:38:05.336 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: jq .osd_mclock_scheduler_client_res 2026-02-13T06:38:05.336 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: bc 2026-02-13T06:38:05.337 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: get_asok_path osd.0 2026-02-13T06:38:05.337 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:05.337 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:05.337 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:05.337 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:05.337 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:05.337 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:05.337 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: CEPH_ARGS= 2026-02-13T06:38:05.337 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:38:05.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:333: TEST_profile_disallow_builtin_params_modify: local opt_val_orig=0.300000 2026-02-13T06:38:05.385 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:334: TEST_profile_disallow_builtin_params_modify: echo '0.300000 + 0.1' 2026-02-13T06:38:05.385 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:334: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:05.386 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:334: TEST_profile_disallow_builtin_params_modify: local opt_val_new=.400000 2026-02-13T06:38:05.386 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:335: TEST_profile_disallow_builtin_params_modify: ceph config set osd.0 osd_mclock_scheduler_client_res .400000 2026-02-13T06:38:05.612 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:338: TEST_profile_disallow_builtin_params_modify: expr 10 - 1 2026-02-13T06:38:05.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:338: TEST_profile_disallow_builtin_params_modify: seq 0 9 2026-02-13T06:38:05.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:338: TEST_profile_disallow_builtin_params_modify: for count in $(seq 0 $(expr $retries - 1)) 2026-02-13T06:38:05.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:340: TEST_profile_disallow_builtin_params_modify: errors=0 2026-02-13T06:38:05.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:341: TEST_profile_disallow_builtin_params_modify: sleep 2 2026-02-13T06:38:07.615 INFO:tasks.workunit.client.0.trial040.stdout:Check configuration values - Attempt#: 0 2026-02-13T06:38:07.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:343: TEST_profile_disallow_builtin_params_modify: echo 'Check configuration values - Attempt#: 0' 2026-02-13T06:38:07.616 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:345: TEST_profile_disallow_builtin_params_modify: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T06:38:07.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:345: TEST_profile_disallow_builtin_params_modify: local res=0.000000 2026-02-13T06:38:07.844 INFO:tasks.workunit.client.0.trial040.stdout:Mon db (or default): osd.0 osd_mclock_scheduler_client_res = 0.000000 2026-02-13T06:38:07.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:346: TEST_profile_disallow_builtin_params_modify: echo 'Mon db (or default): osd.0 osd_mclock_scheduler_client_res = 0.000000' 2026-02-13T06:38:07.844 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:347: TEST_profile_disallow_builtin_params_modify: echo '0.000000 == .400000' 2026-02-13T06:38:07.844 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:347: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:07.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:347: TEST_profile_disallow_builtin_params_modify: (( 0 )) 2026-02-13T06:38:07.846 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: ceph config show osd.0 2026-02-13T06:38:07.846 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: grep osd_mclock_scheduler_client_res 2026-02-13T06:38:07.846 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: awk '{ print $2 }' 2026-02-13T06:38:07.846 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: bc 2026-02-13T06:38:08.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:353: TEST_profile_disallow_builtin_params_modify: res=.300000 2026-02-13T06:38:08.070 INFO:tasks.workunit.client.0.trial040.stdout:Running config: osd.0 osd_mclock_scheduler_client_res = .300000 2026-02-13T06:38:08.070 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:354: TEST_profile_disallow_builtin_params_modify: echo 'Running config: osd.0 osd_mclock_scheduler_client_res = .300000' 2026-02-13T06:38:08.070 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: echo '.300000 == .400000' 2026-02-13T06:38:08.070 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:08.072 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: echo '.300000 != 0.300000' 2026-02-13T06:38:08.072 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:08.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:356: TEST_profile_disallow_builtin_params_modify: (( 0 || 0 )) 2026-02-13T06:38:08.074 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: jq .osd_mclock_scheduler_client_res 2026-02-13T06:38:08.074 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: get_asok_path osd.0 2026-02-13T06:38:08.074 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:08.074 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: bc 2026-02-13T06:38:08.075 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:08.075 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:08.075 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.075 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.075 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:08.075 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: CEPH_ARGS= 2026-02-13T06:38:08.076 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:38:08.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:362: TEST_profile_disallow_builtin_params_modify: res=0.300000 2026-02-13T06:38:08.123 INFO:tasks.workunit.client.0.trial040.stdout:Values map: osd.0 osd_mclock_scheduler_client_res = 0.300000 2026-02-13T06:38:08.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:363: TEST_profile_disallow_builtin_params_modify: echo 'Values map: osd.0 osd_mclock_scheduler_client_res = 0.300000' 2026-02-13T06:38:08.124 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: echo '0.300000 == .400000' 2026-02-13T06:38:08.124 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:08.125 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: echo '0.300000 != 0.300000' 2026-02-13T06:38:08.125 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: bc -l 2026-02-13T06:38:08.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:365: TEST_profile_disallow_builtin_params_modify: (( 0 || 0 )) 2026-02-13T06:38:08.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:370: TEST_profile_disallow_builtin_params_modify: '[' 0 -eq 0 ']' 2026-02-13T06:38:08.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:372: TEST_profile_disallow_builtin_params_modify: break 2026-02-13T06:38:08.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:380: TEST_profile_disallow_builtin_params_modify: teardown td/mclock-config 2026-02-13T06:38:08.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:08.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:08.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:08.128 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:08.128 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:08.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:08.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:08.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:08.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:08.233 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:08.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:08.235 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:08.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:08.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:08.236 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:08.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:08.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:08.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:08.238 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:08.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:08.240 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:08.240 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:08.243 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:08.243 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.243 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.243 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:08.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:08.246 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:08.246 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:08.246 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:08.247 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:08.247 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:08.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:08.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:08.248 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:08.249 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:08.249 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:08.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:08.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:08.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:08.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:08.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:08.252 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:08.253 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.253 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.253 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:08.254 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:08.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:08.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:08.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:08.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:08.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:08.257 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:08.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:08.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:08.258 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:08.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:08.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:08.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:08.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:08.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:08.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:08.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:08.262 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:08.262 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.262 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:08.263 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:08.263 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:08.263 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:38:08.264 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:38:08.264 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.264 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:38:08.265 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:38:08.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:38:08.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:38:08.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:38:08.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_disallow_builtin_params_override td/mclock-config 2026-02-13T06:38:08.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:384: TEST_profile_disallow_builtin_params_override: local dir=td/mclock-config 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:386: TEST_profile_disallow_builtin_params_override: setup td/mclock-config 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:08.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:08.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:08.269 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:08.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:08.269 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:08.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:08.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:08.270 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:08.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:08.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:08.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:08.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:08.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:08.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:08.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:08.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:08.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.274 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:08.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:08.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:08.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:38:08.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:38:08.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.276 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:38:08.277 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:38:08.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:38:08.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:38:08.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:38:08.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:387: TEST_profile_disallow_builtin_params_override: run_mon td/mclock-config a 2026-02-13T06:38:08.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T06:38:08.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:38:08.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:38:08.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:38:08.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T06:38:08.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T06:38:08.297 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:38:08.297 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:08.297 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:08.297 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:08.297 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.298 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.298 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:08.298 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:38:08.323 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:38:08.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:38:08.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:38:08.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:38:08.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:38:08.326 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:38:08.326 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:38:08.326 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:38:08.326 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:38:08.326 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:08.326 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.326 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.326 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:38:08.326 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:38:08.327 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get fsid 2026-02-13T06:38:08.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:38:08.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:38:08.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:38:08.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:38:08.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:38:08.371 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:38:08.371 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:38:08.371 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:38:08.371 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:08.372 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.372 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.372 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:38:08.372 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:38:08.372 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get mon_host 2026-02-13T06:38:08.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:388: TEST_profile_disallow_builtin_params_override: run_mgr td/mclock-config x 2026-02-13T06:38:08.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T06:38:08.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:38:08.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:38:08.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:38:08.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T06:38:08.415 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:38:08.513 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:38:08.513 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:08.513 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:08.513 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:08.513 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.513 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.513 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:08.514 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:38:08.514 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:38:08.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:390: TEST_profile_disallow_builtin_params_override: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T06:38:08.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T06:38:08.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:38:08.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:38:08.535 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T06:38:08.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:38:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:38:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:38:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:38:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:38:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T06:38:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T06:38:08.537 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:38:08.538 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e621309d-bd77-4120-b14c-60c882042f3c 2026-02-13T06:38:08.538 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e621309d-bd77-4120-b14c-60c882042f3c' 2026-02-13T06:38:08.538 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 e621309d-bd77-4120-b14c-60c882042f3c 2026-02-13T06:38:08.538 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:38:08.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDQxo5pZRuAIBAASlW9NUiO65x5Wp6ejY3XYQ== 2026-02-13T06:38:08.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDQxo5pZRuAIBAASlW9NUiO65x5Wp6ejY3XYQ=="}' 2026-02-13T06:38:08.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e621309d-bd77-4120-b14c-60c882042f3c -i td/mclock-config/0/new.json 2026-02-13T06:38:08.650 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:08.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T06:38:08.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQDQxo5pZRuAIBAASlW9NUiO65x5Wp6ejY3XYQ== --osd-uuid e621309d-bd77-4120-b14c-60c882042f3c 2026-02-13T06:38:08.675 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:08.674+0000 7f10eecee900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:08.676 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:08.675+0000 7f10eecee900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:08.677 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:08.676+0000 7f10eecee900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:08.678 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:08.676+0000 7f10eecee900 -1 bdev(0x5617845e4c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:38:08.678 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:08.677+0000 7f10eecee900 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T06:38:09.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T06:38:09.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:38:09.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:38:09.107 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:38:09.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:38:09.212 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:38:09.212 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:38:09.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T06:38:09.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:38:09.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:38:09.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:38:09.229 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:09.228+0000 7f71237c1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:09.231 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:09.229+0000 7f71237c1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:09.232 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:09.230+0000 7f71237c1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:09.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:38:09.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:38:09.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:38:09.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:38:09.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:38:09.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:09.313 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:09.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:38:09.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:09.314 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:38:09.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:38:09.443 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:09.441+0000 7f71237c1900 -1 Falling back to public interface 2026-02-13T06:38:09.762 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:09.759+0000 7f71237c1900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:38:10.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:38:10.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:10.411 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:38:10.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:38:10.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:10.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:38:10.507 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1814605697,v1:127.0.0.1:6801/1814605697] [v2:127.0.0.1:6802/1814605697,v1:127.0.0.1:6803/1814605697] exists,up e621309d-bd77-4120-b14c-60c882042f3c 2026-02-13T06:38:10.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:38:10.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:38:10.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:38:10.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:393: TEST_profile_disallow_builtin_params_override: ceph config get osd.0 osd_mclock_profile 2026-02-13T06:38:10.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:393: TEST_profile_disallow_builtin_params_override: local def_mclock_profile=balanced 2026-02-13T06:38:10.613 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:394: TEST_profile_disallow_builtin_params_override: test balanced = balanced 2026-02-13T06:38:10.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:399: TEST_profile_disallow_builtin_params_override: jq .osd_mclock_profile 2026-02-13T06:38:10.614 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:399: TEST_profile_disallow_builtin_params_override: get_asok_path osd.0 2026-02-13T06:38:10.614 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:10.614 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:10.615 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:10.615 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:10.615 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:10.615 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:10.615 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:399: TEST_profile_disallow_builtin_params_override: CEPH_ARGS= 2026-02-13T06:38:10.615 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:399: TEST_profile_disallow_builtin_params_override: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T06:38:10.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:399: TEST_profile_disallow_builtin_params_override: local 'cur_mclock_profile="high_recovery_ops"' 2026-02-13T06:38:10.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:400: TEST_profile_disallow_builtin_params_override: eval echo '"high_recovery_ops"' 2026-02-13T06:38:10.666 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:400: TEST_profile_disallow_builtin_params_override: echo high_recovery_ops 2026-02-13T06:38:10.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:400: TEST_profile_disallow_builtin_params_override: cur_mclock_profile=high_recovery_ops 2026-02-13T06:38:10.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:401: TEST_profile_disallow_builtin_params_override: test high_recovery_ops = high_recovery_ops 2026-02-13T06:38:10.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:403: TEST_profile_disallow_builtin_params_override: options=('osd_mclock_scheduler_background_recovery_res' 'osd_mclock_scheduler_client_res') 2026-02-13T06:38:10.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:403: TEST_profile_disallow_builtin_params_override: declare -a options 2026-02-13T06:38:10.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:406: TEST_profile_disallow_builtin_params_override: local retries=10 2026-02-13T06:38:10.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:407: TEST_profile_disallow_builtin_params_override: local errors=0 2026-02-13T06:38:10.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:408: TEST_profile_disallow_builtin_params_override: for opt in "${options[@]}" 2026-02-13T06:38:10.667 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: jq .osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:10.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: bc 2026-02-13T06:38:10.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: get_asok_path osd.0 2026-02-13T06:38:10.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:10.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:10.668 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:10.668 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:10.668 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:10.669 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:10.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: CEPH_ARGS= 2026-02-13T06:38:10.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:10.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: local opt_val_orig=0.700000 2026-02-13T06:38:10.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:413: TEST_profile_disallow_builtin_params_override: echo '0.700000 + 0.1' 2026-02-13T06:38:10.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:413: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:10.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:413: TEST_profile_disallow_builtin_params_override: local opt_val_new=.800000 2026-02-13T06:38:10.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:414: TEST_profile_disallow_builtin_params_override: ceph tell osd.0 config set osd_mclock_scheduler_background_recovery_res .800000 2026-02-13T06:38:10.775 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:38:10.775 INFO:tasks.workunit.client.0.trial040.stdout: "success": "osd_mclock_scheduler_background_recovery_res = '' " 2026-02-13T06:38:10.775 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:38:10.781 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:417: TEST_profile_disallow_builtin_params_override: expr 10 - 1 2026-02-13T06:38:10.781 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:417: TEST_profile_disallow_builtin_params_override: seq 0 9 2026-02-13T06:38:10.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:417: TEST_profile_disallow_builtin_params_override: for count in $(seq 0 $(expr $retries - 1)) 2026-02-13T06:38:10.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:419: TEST_profile_disallow_builtin_params_override: errors=0 2026-02-13T06:38:10.783 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:420: TEST_profile_disallow_builtin_params_override: sleep 2 2026-02-13T06:38:12.783 INFO:tasks.workunit.client.0.trial040.stdout:Check configuration values - Attempt#: 0 2026-02-13T06:38:12.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:422: TEST_profile_disallow_builtin_params_override: echo 'Check configuration values - Attempt#: 0' 2026-02-13T06:38:12.784 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:424: TEST_profile_disallow_builtin_params_override: ceph config get osd.0 osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:13.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:424: TEST_profile_disallow_builtin_params_override: local res=0.000000 2026-02-13T06:38:13.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:425: TEST_profile_disallow_builtin_params_override: echo 'Mon db (or default): osd.0 osd_mclock_scheduler_background_recovery_res = 0.000000' 2026-02-13T06:38:13.007 INFO:tasks.workunit.client.0.trial040.stdout:Mon db (or default): osd.0 osd_mclock_scheduler_background_recovery_res = 0.000000 2026-02-13T06:38:13.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:426: TEST_profile_disallow_builtin_params_override: echo '0.000000 == .800000' 2026-02-13T06:38:13.008 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:426: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:13.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:426: TEST_profile_disallow_builtin_params_override: (( 0 )) 2026-02-13T06:38:13.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: ceph config show osd.0 2026-02-13T06:38:13.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: grep osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:13.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: awk '{ print $2 }' 2026-02-13T06:38:13.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: bc 2026-02-13T06:38:13.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: res=.700000 2026-02-13T06:38:13.236 INFO:tasks.workunit.client.0.trial040.stdout:Running config: osd.0 osd_mclock_scheduler_background_recovery_res = .700000 2026-02-13T06:38:13.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:433: TEST_profile_disallow_builtin_params_override: echo 'Running config: osd.0 osd_mclock_scheduler_background_recovery_res = .700000' 2026-02-13T06:38:13.237 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: echo '.700000 == .800000' 2026-02-13T06:38:13.237 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:13.238 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: echo '.700000 != 0.700000' 2026-02-13T06:38:13.238 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:13.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: (( 0 || 0 )) 2026-02-13T06:38:13.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: jq .osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:13.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: bc 2026-02-13T06:38:13.241 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: get_asok_path osd.0 2026-02-13T06:38:13.241 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:13.241 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:13.241 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:13.241 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:13.241 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:13.242 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:13.242 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: CEPH_ARGS= 2026-02-13T06:38:13.242 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_background_recovery_res 2026-02-13T06:38:13.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: res=0.700000 2026-02-13T06:38:13.290 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:442: TEST_profile_disallow_builtin_params_override: echo 'Values map: osd.0 osd_mclock_scheduler_background_recovery_res = 0.700000' 2026-02-13T06:38:13.290 INFO:tasks.workunit.client.0.trial040.stdout:Values map: osd.0 osd_mclock_scheduler_background_recovery_res = 0.700000 2026-02-13T06:38:13.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: echo '0.700000 == .800000' 2026-02-13T06:38:13.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:13.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: echo '0.700000 != 0.700000' 2026-02-13T06:38:13.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:13.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: (( 0 || 0 )) 2026-02-13T06:38:13.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:449: TEST_profile_disallow_builtin_params_override: '[' 0 -eq 0 ']' 2026-02-13T06:38:13.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:451: TEST_profile_disallow_builtin_params_override: break 2026-02-13T06:38:13.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:408: TEST_profile_disallow_builtin_params_override: for opt in "${options[@]}" 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: jq .osd_mclock_scheduler_client_res 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: bc 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: get_asok_path osd.0 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: CEPH_ARGS= 2026-02-13T06:38:13.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:38:13.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:412: TEST_profile_disallow_builtin_params_override: local opt_val_orig=0.300000 2026-02-13T06:38:13.343 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:413: TEST_profile_disallow_builtin_params_override: echo '0.300000 + 0.1' 2026-02-13T06:38:13.343 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:413: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:13.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:413: TEST_profile_disallow_builtin_params_override: local opt_val_new=.400000 2026-02-13T06:38:13.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:414: TEST_profile_disallow_builtin_params_override: ceph tell osd.0 config set osd_mclock_scheduler_client_res .400000 2026-02-13T06:38:13.402 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:38:13.402 INFO:tasks.workunit.client.0.trial040.stdout: "success": "osd_mclock_scheduler_client_res = '' " 2026-02-13T06:38:13.403 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:38:13.408 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:417: TEST_profile_disallow_builtin_params_override: expr 10 - 1 2026-02-13T06:38:13.409 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:417: TEST_profile_disallow_builtin_params_override: seq 0 9 2026-02-13T06:38:13.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:417: TEST_profile_disallow_builtin_params_override: for count in $(seq 0 $(expr $retries - 1)) 2026-02-13T06:38:13.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:419: TEST_profile_disallow_builtin_params_override: errors=0 2026-02-13T06:38:13.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:420: TEST_profile_disallow_builtin_params_override: sleep 2 2026-02-13T06:38:15.411 INFO:tasks.workunit.client.0.trial040.stdout:Check configuration values - Attempt#: 0 2026-02-13T06:38:15.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:422: TEST_profile_disallow_builtin_params_override: echo 'Check configuration values - Attempt#: 0' 2026-02-13T06:38:15.412 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:424: TEST_profile_disallow_builtin_params_override: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T06:38:15.645 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:424: TEST_profile_disallow_builtin_params_override: local res=0.000000 2026-02-13T06:38:15.645 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:425: TEST_profile_disallow_builtin_params_override: echo 'Mon db (or default): osd.0 osd_mclock_scheduler_client_res = 0.000000' 2026-02-13T06:38:15.646 INFO:tasks.workunit.client.0.trial040.stdout:Mon db (or default): osd.0 osd_mclock_scheduler_client_res = 0.000000 2026-02-13T06:38:15.646 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:426: TEST_profile_disallow_builtin_params_override: echo '0.000000 == .400000' 2026-02-13T06:38:15.646 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:426: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:15.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:426: TEST_profile_disallow_builtin_params_override: (( 0 )) 2026-02-13T06:38:15.647 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: ceph config show osd.0 2026-02-13T06:38:15.647 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: grep osd_mclock_scheduler_client_res 2026-02-13T06:38:15.647 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: awk '{ print $2 }' 2026-02-13T06:38:15.648 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: bc 2026-02-13T06:38:15.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:432: TEST_profile_disallow_builtin_params_override: res=.300000 2026-02-13T06:38:15.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:433: TEST_profile_disallow_builtin_params_override: echo 'Running config: osd.0 osd_mclock_scheduler_client_res = .300000' 2026-02-13T06:38:15.879 INFO:tasks.workunit.client.0.trial040.stdout:Running config: osd.0 osd_mclock_scheduler_client_res = .300000 2026-02-13T06:38:15.880 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: echo '.300000 == .400000' 2026-02-13T06:38:15.880 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:15.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: echo '.300000 != 0.300000' 2026-02-13T06:38:15.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:15.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:435: TEST_profile_disallow_builtin_params_override: (( 0 || 0 )) 2026-02-13T06:38:15.883 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: jq .osd_mclock_scheduler_client_res 2026-02-13T06:38:15.883 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: bc 2026-02-13T06:38:15.883 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: get_asok_path osd.0 2026-02-13T06:38:15.883 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:15.883 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:15.884 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:15.884 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:15.884 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:15.884 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:15.884 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: CEPH_ARGS= 2026-02-13T06:38:15.884 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T06:38:15.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:441: TEST_profile_disallow_builtin_params_override: res=0.300000 2026-02-13T06:38:15.932 INFO:tasks.workunit.client.0.trial040.stdout:Values map: osd.0 osd_mclock_scheduler_client_res = 0.300000 2026-02-13T06:38:15.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:442: TEST_profile_disallow_builtin_params_override: echo 'Values map: osd.0 osd_mclock_scheduler_client_res = 0.300000' 2026-02-13T06:38:15.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: echo '0.300000 == .400000' 2026-02-13T06:38:15.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:15.934 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: echo '0.300000 != 0.300000' 2026-02-13T06:38:15.934 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: bc -l 2026-02-13T06:38:15.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:444: TEST_profile_disallow_builtin_params_override: (( 0 || 0 )) 2026-02-13T06:38:15.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:449: TEST_profile_disallow_builtin_params_override: '[' 0 -eq 0 ']' 2026-02-13T06:38:15.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:451: TEST_profile_disallow_builtin_params_override: break 2026-02-13T06:38:15.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:459: TEST_profile_disallow_builtin_params_override: teardown td/mclock-config 2026-02-13T06:38:15.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:15.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:15.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:15.937 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:15.937 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:15.937 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:15.937 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:15.937 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:16.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:16.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:16.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:16.044 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:16.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:16.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:16.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:16.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:16.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:16.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:16.047 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:16.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:16.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:16.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:16.052 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:16.052 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.052 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.053 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:16.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:16.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:16.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T06:38:16.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:16.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:16.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:16.055 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:16.055 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:16.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:16.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:16.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:16.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:16.056 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:16.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:16.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:16.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:16.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:16.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:16.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:16.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:16.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:16.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:16.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:16.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:16.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:16.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:16.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:16.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:16.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:16.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T06:38:16.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T06:38:16.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:38:16.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:38:16.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:16.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:16.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:16.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:16.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:16.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:16.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:16.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:16.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:16.066 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:16.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:16.067 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:16.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:16.068 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:16.068 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:16.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:16.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:16.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:16.069 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:16.070 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:16.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:16.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:16.072 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:16.072 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.072 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:16.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:16.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:16.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:38:16.074 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:38:16.074 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.074 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:38:16.075 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_recovery_limit_adjustment_mclock td/mclock-config 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:204: TEST_recovery_limit_adjustment_mclock: setup td/mclock-config 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:16.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:16.078 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:16.078 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:16.078 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:16.078 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:16.078 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:16.078 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:16.078 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:16.079 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:16.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:16.079 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:16.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:16.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:16.080 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:16.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:16.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:16.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:16.082 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:16.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:16.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:16.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:16.084 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:16.084 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.084 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:16.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:16.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:16.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T06:38:16.086 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:38:16.086 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.087 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19092 2026-02-13T06:38:16.087 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:205: TEST_recovery_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T06:38:16.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T06:38:16.107 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:38:16.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:16.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:16.108 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:16.108 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.108 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:16.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:38:16.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:38:16.135 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:38:16.135 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:38:16.135 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:38:16.135 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:38:16.136 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:38:16.136 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:38:16.136 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:38:16.136 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:38:16.136 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:16.137 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.137 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.137 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:38:16.137 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:38:16.137 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get fsid 2026-02-13T06:38:16.179 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:38:16.179 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:38:16.180 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:38:16.180 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:38:16.180 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:38:16.181 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:38:16.181 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:38:16.181 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:38:16.181 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:16.181 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.181 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.181 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-mon.a.asok 2026-02-13T06:38:16.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:38:16.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19092/ceph-mon.a.asok config get mon_host 2026-02-13T06:38:16.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:206: TEST_recovery_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-13T06:38:16.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T06:38:16.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:38:16.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:38:16.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:38:16.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T06:38:16.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:38:16.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:38:16.325 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:16.325 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:16.325 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:16.325 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.325 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.325 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:16.325 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:38:16.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:38:16.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:208: TEST_recovery_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T06:38:16.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:38:16.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:38:16.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T06:38:16.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T06:38:16.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:38:16.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:38:16.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:38:16.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:38:16.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:38:16.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T06:38:16.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T06:38:16.349 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:38:16.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=92a8fcb8-a327-402a-b1ca-c8ff320c3dc2 2026-02-13T06:38:16.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 92a8fcb8-a327-402a-b1ca-c8ff320c3dc2' 2026-02-13T06:38:16.350 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 92a8fcb8-a327-402a-b1ca-c8ff320c3dc2 2026-02-13T06:38:16.350 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:38:16.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDYxo5pOmlXFRAASzQco99/jdwqst8qfWpQdA== 2026-02-13T06:38:16.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDYxo5pOmlXFRAASzQco99/jdwqst8qfWpQdA=="}' 2026-02-13T06:38:16.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 92a8fcb8-a327-402a-b1ca-c8ff320c3dc2 -i td/mclock-config/0/new.json 2026-02-13T06:38:16.463 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:16.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T06:38:16.471 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQDYxo5pOmlXFRAASzQco99/jdwqst8qfWpQdA== --osd-uuid 92a8fcb8-a327-402a-b1ca-c8ff320c3dc2 2026-02-13T06:38:16.489 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:16.486+0000 7f3488022900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:16.490 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:16.488+0000 7f3488022900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:16.491 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:16.489+0000 7f3488022900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:16.492 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:16.489+0000 7f3488022900 -1 bdev(0x558e36088c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:38:16.492 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:16.489+0000 7f3488022900 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T06:38:16.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T06:38:16.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:38:16.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:38:16.948 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:38:16.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:38:17.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:38:17.054 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:38:17.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8446ef84-f1f2-4690-9c57-05fc6ec640b9 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.19092/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T06:38:17.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:38:17.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:38:17.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:38:17.071 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:17.068+0000 7f49e5825900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:17.072 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:17.070+0000 7f49e5825900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:17.073 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:17.071+0000 7f49e5825900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:17.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:38:17.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:38:17.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:38:17.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:38:17.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:38:17.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:17.152 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:17.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:38:17.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:17.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:38:17.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:38:17.298 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:17.296+0000 7f49e5825900 -1 Falling back to public interface 2026-02-13T06:38:17.780 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:17.778+0000 7f49e5825900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:38:18.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:38:18.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:18.252 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:38:18.252 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:38:18.253 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:18.253 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:38:18.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:38:19.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:38:19.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:19.353 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T06:38:19.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-13T06:38:19.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:19.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:38:19.578 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3328097517,v1:127.0.0.1:6801/3328097517] [v2:127.0.0.1:6802/3328097517,v1:127.0.0.1:6803/3328097517] exists,up 92a8fcb8-a327-402a-b1ca-c8ff320c3dc2 2026-02-13T06:38:19.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:38:19.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:38:19.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:38:19.580 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:38:19.580 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:19.580 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:19.580 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:19.581 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:19.581 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:19.581 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:19.581 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:38:19.581 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-13T06:38:19.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: local 'recoveries={"osd_recovery_max_active":"0"}' 2026-02-13T06:38:19.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: echo '{"osd_recovery_max_active":"0"}' 2026-02-13T06:38:19.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: grep --quiet osd_recovery_max_active 2026-02-13T06:38:19.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:217: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-13T06:38:19.859 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:218: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-13T06:38:21.862 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:38:21.862 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:21.863 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:21.863 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:21.863 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:21.863 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:21.863 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:21.863 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:38:21.864 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-13T06:38:21.912 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: local 'max_recoveries={"osd_recovery_max_active":"0"}' 2026-02-13T06:38:21.912 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"0"}' = '{"osd_recovery_max_active":"0"}' 2026-02-13T06:38:21.912 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:225: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-13T06:38:22.140 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:226: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-13T06:38:22.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:227: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-13T06:38:24.370 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T06:38:24.370 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:38:24.370 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:38:24.370 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:24.370 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.370 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:24.370 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19092/ceph-osd.0.asok 2026-02-13T06:38:24.371 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T06:38:24.371 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.19092/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-13T06:38:24.419 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: max_recoveries='{"osd_recovery_max_active":"10"}' 2026-02-13T06:38:24.419 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:230: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"10"}' = '{"osd_recovery_max_active":"10"}' 2026-02-13T06:38:24.419 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:232: TEST_recovery_limit_adjustment_mclock: teardown td/mclock-config 2026-02-13T06:38:24.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:24.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:24.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:24.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:24.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:24.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:24.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:24.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:24.527 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:24.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:24.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:24.528 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:24.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:24.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:24.530 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:24.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:24.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:24.532 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:24.532 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:24.532 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:24.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:24.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:24.537 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:24.537 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.537 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:24.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:24.538 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:24.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:24.540 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:24.540 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:24.541 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:24.541 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:24.542 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:24.542 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:24.543 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:24.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:24.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:24.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:24.544 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:24.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:24.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:24.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:24.546 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:24.546 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.547 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:24.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:24.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:24.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:24.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-13T06:38:24.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/mclock-config 0 2026-02-13T06:38:24.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T06:38:24.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T06:38:24.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T06:38:24.549 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:24.549 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:24.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:24.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:24.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:24.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:24.550 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:24.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:24.551 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:24.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:24.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:24.552 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:24.553 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:24.553 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:24.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:24.554 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:24.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:24.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T06:38:24.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T06:38:24.556 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:24.556 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.557 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19092 2026-02-13T06:38:24.557 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19092 2026-02-13T06:38:24.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:24.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:24.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-13T06:38:24.559 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T06:38:24.559 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T06:38:24.626 INFO:tasks.workunit:Running workunit misc/network-ping.sh... 2026-02-13T06:38:24.626 DEBUG:teuthology.orchestra.run.trial040:workunit test misc/network-ping.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh 2026-02-13T06:38:24.689 INFO:tasks.workunit.client.0.trial040.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T06:38:24.693 INFO:tasks.workunit.client.0.trial040.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T06:38:24.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:38:24.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/network-ping 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:6: run: local dir=td/network-ping 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:7: run: shift 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: export CEPH_MON=127.0.0.1:7146 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: CEPH_MON=127.0.0.1:7146 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:10: run: export CEPH_ARGS 2026-02-13T06:38:24.694 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: uuidgen 2026-02-13T06:38:24.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: CEPH_ARGS+='--fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none ' 2026-02-13T06:38:24.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:12: run: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-13T06:38:24.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:13: run: CEPH_ARGS+='--debug_disable_randomized_ping=true ' 2026-02-13T06:38:24.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:14: run: CEPH_ARGS+='--debug_heartbeat_testing_span=5 ' 2026-02-13T06:38:24.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:15: run: CEPH_ARGS+='--osd_heartbeat_interval=1 ' 2026-02-13T06:38:24.695 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: set 2026-02-13T06:38:24.695 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: local 'funcs=TEST_network_ping_test1 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:TEST_network_ping_test2' 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:38:24.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:38:24.698 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:38:24.698 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:38:24.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:38:24.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:38:24.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:38:24.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:38:24.700 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:38:24.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:24.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:38:24.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:38:24.701 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:38:24.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:38:24.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:38:24.703 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T06:38:24.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:38:24.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:38:24.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.33109 2026-02-13T06:38:24.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:38:24.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:38:24.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-13T06:38:24.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:38:24.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:38:24.706 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.33109 2026-02-13T06:38:24.707 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:38:24.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T06:38:24.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T06:38:24.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:38:24.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-13T06:38:24.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test1 td/network-ping 2026-02-13T06:38:24.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:25: TEST_network_ping_test1: local dir=td/network-ping 2026-02-13T06:38:24.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:27: TEST_network_ping_test1: run_mon td/network-ping a 2026-02-13T06:38:24.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-13T06:38:24.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:38:24.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:38:24.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:38:24.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-13T06:38:24.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-13T06:38:24.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:38:24.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:24.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:24.729 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:24.729 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.729 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:38:24.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:38:24.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:38:24.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:38:24.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:38:24.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:38:24.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:38:24.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:38:24.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:38:24.757 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:38:24.757 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:38:24.757 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:38:24.758 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:24.758 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.759 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:38:24.759 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mon.a.asok 2026-02-13T06:38:24.759 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:38:24.759 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.33109/ceph-mon.a.asok config get fsid 2026-02-13T06:38:24.801 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:38:24.801 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:38:24.801 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:38:24.801 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:38:24.802 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:38:24.803 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:38:24.803 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:38:24.803 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:38:24.803 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:38:24.803 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.803 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:38:24.803 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mon.a.asok 2026-02-13T06:38:24.803 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:38:24.803 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.33109/ceph-mon.a.asok config get mon_host 2026-02-13T06:38:24.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:28: TEST_network_ping_test1: run_mgr td/network-ping x 2026-02-13T06:38:24.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-13T06:38:24.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:38:24.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:38:24.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:38:24.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-13T06:38:24.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:38:24.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:38:24.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:24.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:24.945 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:24.945 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.945 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:38:24.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:38:24.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:38:24.946 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:38:24.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:29: TEST_network_ping_test1: run_osd td/network-ping 0 2026-02-13T06:38:24.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-13T06:38:24.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:38:24.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=718fd47b-0096-4551-80db-a7cb898f67e4 2026-02-13T06:38:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 718fd47b-0096-4551-80db-a7cb898f67e4' 2026-02-13T06:38:24.970 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 718fd47b-0096-4551-80db-a7cb898f67e4 2026-02-13T06:38:24.970 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:38:24.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgxo5pI5E9OhAAPaQ/Vx5KiHN2UeTg4HJ+Nw== 2026-02-13T06:38:24.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgxo5pI5E9OhAAPaQ/Vx5KiHN2UeTg4HJ+Nw=="}' 2026-02-13T06:38:24.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 718fd47b-0096-4551-80db-a7cb898f67e4 -i td/network-ping/0/new.json 2026-02-13T06:38:25.082 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:25.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-13T06:38:25.090 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgxo5pI5E9OhAAPaQ/Vx5KiHN2UeTg4HJ+Nw== --osd-uuid 718fd47b-0096-4551-80db-a7cb898f67e4 2026-02-13T06:38:25.109 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:25.106+0000 7f5be5e19900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:25.110 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:25.108+0000 7f5be5e19900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:25.111 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:25.109+0000 7f5be5e19900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:25.112 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:25.109+0000 7f5be5e19900 -1 bdev(0x55dc98a7cc00 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:38:25.112 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:25.109+0000 7f5be5e19900 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-13T06:38:25.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-13T06:38:25.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:38:25.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:38:25.518 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:38:25.519 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:38:25.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:38:25.624 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:38:25.624 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:38:25.624 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:38:25.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:38:25.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:38:25.641 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:25.639+0000 7f8fd4a72900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:25.643 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:25.641+0000 7f8fd4a72900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:25.644 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:25.642+0000 7f8fd4a72900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:25.722 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:38:25.722 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:38:25.722 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:38:25.722 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:38:25.722 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:38:25.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:25.723 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:25.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:38:25.724 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:25.724 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:38:25.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:38:25.843 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:25.841+0000 7f8fd4a72900 -1 Falling back to public interface 2026-02-13T06:38:26.163 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:26.160+0000 7f8fd4a72900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:38:26.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:38:26.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:26.821 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:38:26.822 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:38:26.822 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:26.822 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:38:26.918 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/820767214,v1:127.0.0.1:6801/820767214] [v2:127.0.0.1:6802/820767214,v1:127.0.0.1:6803/820767214] exists,up 718fd47b-0096-4551-80db-a7cb898f67e4 2026-02-13T06:38:26.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:38:26.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:38:26.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:38:26.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:30: TEST_network_ping_test1: run_osd td/network-ping 1 2026-02-13T06:38:26.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T06:38:26.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:38:26.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:38:26.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:38:26.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=20ffe127-1c18-4803-857e-53d44595fa3d 2026-02-13T06:38:26.926 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 20ffe127-1c18-4803-857e-53d44595fa3d 2026-02-13T06:38:26.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 20ffe127-1c18-4803-857e-53d44595fa3d' 2026-02-13T06:38:26.926 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:38:26.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDixo5pb3OGNxAAMdhFdJspBk4W1NLRtE8Ufw== 2026-02-13T06:38:26.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDixo5pb3OGNxAAMdhFdJspBk4W1NLRtE8Ufw=="}' 2026-02-13T06:38:26.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 20ffe127-1c18-4803-857e-53d44595fa3d -i td/network-ping/1/new.json 2026-02-13T06:38:27.038 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:38:27.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-13T06:38:27.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDixo5pb3OGNxAAMdhFdJspBk4W1NLRtE8Ufw== --osd-uuid 20ffe127-1c18-4803-857e-53d44595fa3d 2026-02-13T06:38:27.063 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:27.061+0000 7ffb4961c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:27.065 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:27.062+0000 7ffb4961c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:27.066 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:27.063+0000 7ffb4961c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:27.066 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:27.063+0000 7ffb4961c900 -1 bdev(0x55a6c9f3ec00 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-13T06:38:27.066 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:27.064+0000 7ffb4961c900 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-13T06:38:27.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-13T06:38:27.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:38:27.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T06:38:27.397 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T06:38:27.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:38:27.711 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T06:38:27.712 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T06:38:27.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:38:27.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:38:27.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:38:27.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:38:27.728 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:27.726+0000 7fbe9fc0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:27.730 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:27.727+0000 7fbe9fc0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:27.730 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:27.728+0000 7fbe9fc0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:27.864 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:27.862+0000 7fbe9fc0f900 -1 Falling back to public interface 2026-02-13T06:38:27.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T06:38:27.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:38:27.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T06:38:27.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:38:27.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:38:27.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:27.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:38:27.940 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:27.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:27.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:38:28.167 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:38:28.172 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:28.169+0000 7fbe9fc0f900 -1 osd.1 0 log_to_monitors true 2026-02-13T06:38:29.168 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:38:29.169 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:29.169 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:38:29.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:38:29.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:29.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:38:29.397 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2291853422,v1:127.0.0.1:6811/2291853422] [v2:127.0.0.1:6812/2291853422,v1:127.0.0.1:6813/2291853422] exists,up 20ffe127-1c18-4803-857e-53d44595fa3d 2026-02-13T06:38:29.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:38:29.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:38:29.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:38:29.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:31: TEST_network_ping_test1: run_osd td/network-ping 2 2026-02-13T06:38:29.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:38:29.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:38:29.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=88a4742f-3dba-4e2a-b023-85c62c2a0b58 2026-02-13T06:38:29.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 88a4742f-3dba-4e2a-b023-85c62c2a0b58' 2026-02-13T06:38:29.405 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 88a4742f-3dba-4e2a-b023-85c62c2a0b58 2026-02-13T06:38:29.405 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:38:29.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDlxo5pIFSRGBAAwnpTm1GW4ZrWQzTW40HnDQ== 2026-02-13T06:38:29.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDlxo5pIFSRGBAAwnpTm1GW4ZrWQzTW40HnDQ=="}' 2026-02-13T06:38:29.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 88a4742f-3dba-4e2a-b023-85c62c2a0b58 -i td/network-ping/2/new.json 2026-02-13T06:38:29.641 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T06:38:29.649 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-13T06:38:29.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDlxo5pIFSRGBAAwnpTm1GW4ZrWQzTW40HnDQ== --osd-uuid 88a4742f-3dba-4e2a-b023-85c62c2a0b58 2026-02-13T06:38:29.668 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:29.665+0000 7fe99e81f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:29.669 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:29.667+0000 7fe99e81f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:29.670 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:29.668+0000 7fe99e81f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:29.671 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:29.668+0000 7fe99e81f900 -1 bdev(0x56495b9f6c00 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-13T06:38:29.671 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:29.668+0000 7fe99e81f900 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-13T06:38:29.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-13T06:38:29.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:38:29.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T06:38:29.982 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T06:38:29.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:38:30.302 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T06:38:30.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T06:38:30.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:38:30.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:38:30.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:38:30.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:38:30.320 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:30.318+0000 7fc4c2c23900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:30.322 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:30.319+0000 7fc4c2c23900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:30.322 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:30.320+0000 7fc4c2c23900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:38:30.477 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:30.474+0000 7fc4c2c23900 -1 Falling back to public interface 2026-02-13T06:38:30.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T06:38:30.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:38:30.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T06:38:30.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:38:30.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:38:30.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:30.531 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:38:30.532 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:38:30.532 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:30.532 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T06:38:30.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:38:30.810 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:38:30.807+0000 7fc4c2c23900 -1 osd.2 0 log_to_monitors true 2026-02-13T06:38:31.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:38:31.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:38:31.759 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:38:31.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:38:31.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:38:31.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T06:38:31.984 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4287313589,v1:127.0.0.1:6819/4287313589] [v2:127.0.0.1:6820/4287313589,v1:127.0.0.1:6821/4287313589] exists,up 88a4742f-3dba-4e2a-b023-85c62c2a0b58 2026-02-13T06:38:31.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:38:31.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:38:31.986 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:38:31.986 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:33: TEST_network_ping_test1: sleep 5 2026-02-13T06:38:36.986 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:35: TEST_network_ping_test1: create_pool foo 16 2026-02-13T06:38:36.986 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-13T06:38:37.258 INFO:tasks.workunit.client.0.trial040.stderr:pool 'foo' already exists 2026-02-13T06:38:37.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:38:38.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:38: TEST_network_ping_test1: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-13T06:38:38.289 INFO:tasks.workunit.client.0.trial040.stdout:hints = 1 2026-02-13T06:38:38.290 INFO:tasks.workunit.client.0.trial040.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-13T06:38:38.290 INFO:tasks.workunit.client.0.trial040.stdout:Object prefix: benchmark_data_trial040_37594 2026-02-13T06:38:38.290 INFO:tasks.workunit.client.0.trial040.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-13T06:38:38.290 INFO:tasks.workunit.client.0.trial040.stdout: 0 0 0 0 0 0 - 0 2026-02-13T06:38:39.289 INFO:tasks.workunit.client.0.trial040.stdout: 1 16 11312 11296 44.1217 44.125 0.00136476 0.00141424 2026-02-13T06:38:40.289 INFO:tasks.workunit.client.0.trial040.stdout: 2 16 23103 23087 45.0885 46.0586 0.00183293 0.0013825 2026-02-13T06:38:41.289 INFO:tasks.workunit.client.0.trial040.stdout: 3 16 35101 35085 45.6803 46.8672 0.00124458 0.00136667 2026-02-13T06:38:42.289 INFO:tasks.workunit.client.0.trial040.stdout: 4 16 46893 46877 45.775 46.0625 0.00187002 0.00136393 2026-02-13T06:38:43.290 INFO:tasks.workunit.client.0.trial040.stdout: 5 16 58766 58750 45.895 46.3789 0.000960857 0.00136042 2026-02-13T06:38:44.290 INFO:tasks.workunit.client.0.trial040.stdout: 6 16 70947 70931 46.1756 47.582 0.000878373 0.00135212 2026-02-13T06:38:45.290 INFO:tasks.workunit.client.0.trial040.stdout: 7 16 82409 82393 45.9748 44.7734 0.00119062 0.00135805 2026-02-13T06:38:46.290 INFO:tasks.workunit.client.0.trial040.stdout: 8 16 92988 92972 45.393 41.3242 0.00164537 0.00137549 2026-02-13T06:38:47.290 INFO:tasks.workunit.client.0.trial040.stdout: 9 15 103151 103136 44.7605 39.7031 0.00139242 0.00139497 2026-02-13T06:38:48.290 INFO:tasks.workunit.client.0.trial040.stdout: 10 8 113270 113262 44.2396 39.5547 0.00116188 0.00141142 2026-02-13T06:38:49.290 INFO:tasks.workunit.client.0.trial040.stdout:Total time run: 10.0011 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Total writes made: 113270 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Write size: 4096 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Object size: 4096 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Bandwidth (MB/sec): 44.2414 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Stddev Bandwidth: 2.99277 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Max bandwidth (MB/sec): 47.582 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Min bandwidth (MB/sec): 39.5547 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Average IOPS: 11325 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Stddev IOPS: 766.148 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Max IOPS: 12181 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Min IOPS: 10126 2026-02-13T06:38:49.291 INFO:tasks.workunit.client.0.trial040.stdout:Average Latency(s): 0.00141141 2026-02-13T06:38:49.292 INFO:tasks.workunit.client.0.trial040.stdout:Stddev Latency(s): 0.000685177 2026-02-13T06:38:49.292 INFO:tasks.workunit.client.0.trial040.stdout:Max latency(s): 0.012378 2026-02-13T06:38:49.292 INFO:tasks.workunit.client.0.trial040.stdout:Min latency(s): 0.000430099 2026-02-13T06:38:49.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:41: TEST_network_ping_test1: sleep 10 2026-02-13T06:38:59.297 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:42: TEST_network_ping_test1: flush_pg_stats 2026-02-13T06:38:59.297 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:38:59.298 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:38:59.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:38:59.578 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T06:38:59.578 INFO:tasks.workunit.client.0.trial040.stderr:2' 2026-02-13T06:38:59.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:38:59.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:38:59.579 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:38:59.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-13T06:38:59.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-13T06:38:59.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-13T06:38:59.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:38:59.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:38:59.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705673 2026-02-13T06:38:59.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705673 2026-02-13T06:38:59.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705673' 2026-02-13T06:38:59.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:38:59.747 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T06:38:59.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-13T06:38:59.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-13T06:38:59.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705673 2-55834574856' 2026-02-13T06:38:59.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:38:59.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-13T06:38:59.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:38:59.822 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:38:59.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-13T06:38:59.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:38:59.824 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-13T06:38:59.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-13T06:38:59.825 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836489 2026-02-13T06:38:59.825 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:39:00.156 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-13T06:39:00.156 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:39:01.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:39:01.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:39:01.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-13T06:39:01.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:39:01.453 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705673 2026-02-13T06:39:01.454 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:39:01.454 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:39:01.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705673 2026-02-13T06:39:01.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:39:01.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705673 2026-02-13T06:39:01.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-13T06:39:01.457 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 38654705673 2026-02-13T06:39:01.457 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:39:01.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705673 -lt 38654705673 2026-02-13T06:39:01.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:39:01.800 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574856 2026-02-13T06:39:01.800 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:39:01.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T06:39:01.801 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574856 2026-02-13T06:39:01.801 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:39:01.802 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574856 2026-02-13T06:39:01.803 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574856' 2026-02-13T06:39:01.803 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 55834574856 2026-02-13T06:39:01.803 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T06:39:02.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-13T06:39:02.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:39:02.116 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T06:39:02.117 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:39:02.117 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:39:02.117 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:39:02.117 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:39:02.117 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:39:02.117 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-osd.0.asok 2026-02-13T06:39:02.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:39:02.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-osd.0.asok dump_osd_network 2026-02-13T06:39:02.160 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:39:02.160 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 1000, 2026-02-13T06:39:02.160 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [] 2026-02-13T06:39:02.160 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:39:02.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:02.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:39:02.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:39:02.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:02.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: jq .threshold 2026-02-13T06:39:02.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T06:39:02.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:39:02.208 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T06:39:02.208 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T06:39:02.209 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T06:39:02.209 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:39:02.209 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:39:02.209 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:39:02.209 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mgr.x.asok 2026-02-13T06:39:02.209 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:39:02.209 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-mgr.x.asok dump_osd_network 2026-02-13T06:39:02.248 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:39:02.248 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 1000, 2026-02-13T06:39:02.249 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [] 2026-02-13T06:39:02.249 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:39:02.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:02.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:39:02.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:39:02.263 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:02.263 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: jq .threshold 2026-02-13T06:39:02.265 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T06:39:02.265 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:39:02.266 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T06:39:02.266 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:39:02.266 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:39:02.266 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:39:02.267 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:39:02.267 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:39:02.267 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-osd.0.asok 2026-02-13T06:39:02.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:39:02.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-osd.0.asok dump_osd_network 0 2026-02-13T06:39:02.310 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:39:02.310 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 0, 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [ 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:01 2026", 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.117, 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.130, 2026-02-13T06:39:02.311 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.244 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.190, 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.190, 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.156 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.190, 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.190, 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.156 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.312 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.169 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:01 2026", 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.150, 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.232, 2026-02-13T06:39:02.313 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.202 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.199, 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 1.246, 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.246 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.199, 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 1.246, 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.246 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.314 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.150 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:02 2026", 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.153, 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.163, 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.199 2026-02-13T06:39:02.315 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.206, 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.322, 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.510 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.206, 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.322, 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.510 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.193 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.316 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:02 2026", 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.165, 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.162, 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.190 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.200, 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.286, 2026-02-13T06:39:02.317 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.492 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.200, 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.286, 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.492 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.184 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T06:39:02.318 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:39:02.334 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:02.335 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:39:02.336 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: test 4 = 4 2026-02-13T06:39:02.337 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:02.337 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: jq .threshold 2026-02-13T06:39:02.339 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:39:02.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:39:02.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T06:39:02.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T06:39:02.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T06:39:02.341 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:39:02.341 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:39:02.341 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:39:02.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mgr.x.asok 2026-02-13T06:39:02.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:39:02.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-mgr.x.asok dump_osd_network 0 2026-02-13T06:39:02.381 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:39:02.381 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 0, 2026-02-13T06:39:02.381 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [ 2026-02-13T06:39:02.381 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:56 2026", 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.098, 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.167, 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.256 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.382 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.086, 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.086, 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.086 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.110, 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 1.156, 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.156 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.102 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.383 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:56 2026", 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.120, 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.239, 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.204 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.103, 2026-02-13T06:39:02.384 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.103, 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.103 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.136, 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 1.246, 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.246 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.129 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:57 2026", 2026-02-13T06:39:02.385 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.132, 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.176, 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.204 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.112, 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.108, 2026-02-13T06:39:02.386 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.108 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.153, 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.510, 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.510 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.122 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:57 2026", 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.387 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.148, 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.170, 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.193 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.109, 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.109, 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.109 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.388 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.190, 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.492, 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.492 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.146 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:56 2026", 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:02.389 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.115, 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.174, 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.150 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.094, 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.094, 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.094 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.148, 2026-02-13T06:39:02.390 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.442, 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.442 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.176 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:56 2026", 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:02.391 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.143, 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.164, 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.155 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.110, 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.099, 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.099 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.392 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.167, 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.327, 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.327 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.156 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:57 2026", 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:02.393 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.116, 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.158, 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.134 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.088, 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.079, 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.079 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.133, 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.337, 2026-02-13T06:39:02.394 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.337 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.101 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:59 2026", 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.089, 2026-02-13T06:39:02.395 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.122, 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.156 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.078, 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.076, 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.076 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.099, 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.378, 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.445 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.396 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.078 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:56 2026", 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.138, 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.148, 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.150 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.397 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.115, 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.109, 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.109 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.178, 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.307, 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.307 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.163 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:57 2026", 2026-02-13T06:39:02.398 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.140, 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.150, 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.134 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.121, 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.105, 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.105 2026-02-13T06:39:02.399 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.160, 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.218, 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.218 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.096 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:59 2026", 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:39:02.400 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.088, 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.125, 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.124 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.074, 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.074, 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.074 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.401 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.103, 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.203, 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.203 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.088 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:38:56 2026", 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:02.402 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.094, 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.115, 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.122 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.082, 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.080, 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.080 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.120, 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.192, 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.192 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:02.403 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.141 2026-02-13T06:39:02.404 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T06:39:02.404 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T06:39:02.404 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:39:02.405 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:02.405 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:39:02.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: test 12 = 12 2026-02-13T06:39:02.405 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:02.405 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: jq .threshold 2026-02-13T06:39:02.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:39:02.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:61: TEST_network_ping_test1: sleep 20 2026-02-13T06:39:22.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:62: TEST_network_ping_test1: flush_pg_stats 2026-02-13T06:39:22.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:39:22.393 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:39:22.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:39:22.618 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T06:39:22.618 INFO:tasks.workunit.client.0.trial040.stderr:2' 2026-02-13T06:39:22.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:39:22.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:39:22.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:39:22.685 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836496 2026-02-13T06:39:22.685 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836496 2026-02-13T06:39:22.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496' 2026-02-13T06:39:22.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:39:22.686 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:39:22.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705679 2026-02-13T06:39:22.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705679 2026-02-13T06:39:22.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496 1-38654705679' 2026-02-13T06:39:22.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:39:22.750 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T06:39:22.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574863 2026-02-13T06:39:22.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574863 2026-02-13T06:39:22.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496 1-38654705679 2-55834574863' 2026-02-13T06:39:22.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:39:22.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836496 2026-02-13T06:39:22.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:39:22.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:39:22.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836496 2026-02-13T06:39:22.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:39:22.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836496 2026-02-13T06:39:22.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-13T06:39:22.818 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836496 2026-02-13T06:39:22.818 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:39:23.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836496 -lt 21474836496 2026-02-13T06:39:23.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:39:23.044 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705679 2026-02-13T06:39:23.044 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:39:23.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:39:23.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705679 2026-02-13T06:39:23.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:39:23.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705679 2026-02-13T06:39:23.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705679' 2026-02-13T06:39:23.047 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 38654705679 2026-02-13T06:39:23.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:39:23.274 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705679 -lt 38654705679 2026-02-13T06:39:23.274 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:39:23.275 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574863 2026-02-13T06:39:23.275 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:39:23.276 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T06:39:23.277 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574863 2026-02-13T06:39:23.277 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:39:23.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574863 2026-02-13T06:39:23.278 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 55834574863 2026-02-13T06:39:23.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574863' 2026-02-13T06:39:23.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T06:39:23.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574863 -lt 55834574863 2026-02-13T06:39:23.503 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:39:23.503 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T06:39:23.503 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:39:23.503 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:39:23.503 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:39:23.503 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:39:23.504 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:39:23.504 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-osd.0.asok 2026-02-13T06:39:23.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:39:23.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-osd.0.asok dump_osd_network 2026-02-13T06:39:23.548 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:39:23.548 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 1000, 2026-02-13T06:39:23.548 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [] 2026-02-13T06:39:23.548 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:39:23.552 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:23.553 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:39:23.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:39:23.555 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:23.555 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: jq .threshold 2026-02-13T06:39:23.557 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T06:39:23.557 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:39:23.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T06:39:23.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T06:39:23.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T06:39:23.559 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:39:23.559 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:39:23.559 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:39:23.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mgr.x.asok 2026-02-13T06:39:23.559 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:39:23.559 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-mgr.x.asok dump_osd_network 2026-02-13T06:39:23.600 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:39:23.600 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 1000, 2026-02-13T06:39:23.600 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [] 2026-02-13T06:39:23.601 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:39:23.606 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:23.606 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:39:23.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:39:23.609 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:23.609 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: jq .threshold 2026-02-13T06:39:23.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T06:39:23.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:39:23.612 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T06:39:23.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:39:23.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:39:23.613 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:39:23.613 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:39:23.613 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:39:23.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-osd.0.asok 2026-02-13T06:39:23.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:39:23.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-osd.0.asok dump_osd_network 0 2026-02-13T06:39:23.657 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:39:23.657 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 0, 2026-02-13T06:39:23.657 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [ 2026-02-13T06:39:23.657 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.657 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:21 2026", 2026-02-13T06:39:23.657 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.657 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:23.657 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:23.657 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.094, 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.108, 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.192 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.112, 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.190, 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.156 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.112, 2026-02-13T06:39:23.658 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.190, 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.156 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.158 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:21 2026", 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.659 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.140, 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.137, 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.188 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.181, 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.217, 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.246 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.181, 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.217, 2026-02-13T06:39:23.660 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.246 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.189 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:22 2026", 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.119, 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.139, 2026-02-13T06:39:23.661 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.178 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.129, 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.206, 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.510 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.129, 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.206, 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.510 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.178 2026-02-13T06:39:23.662 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:22 2026", 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.123, 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.142, 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.172 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.155, 2026-02-13T06:39:23.663 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.206, 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.492 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.155, 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.206, 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.492 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.170 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T06:39:23.664 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:39:23.665 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:23.665 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:39:23.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: test 4 = 4 2026-02-13T06:39:23.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:23.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: jq .threshold 2026-02-13T06:39:23.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:39:23.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:39:23.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T06:39:23.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T06:39:23.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T06:39:23.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:39:23.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:39:23.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:39:23.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mgr.x.asok 2026-02-13T06:39:23.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:39:23.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-mgr.x.asok dump_osd_network 0 2026-02-13T06:39:23.706 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:39:23.706 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 0, 2026-02-13T06:39:23.706 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [ 2026-02-13T06:39:23.706 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:21 2026", 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.094, 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.108, 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.192 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.707 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.076, 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.076, 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.076 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.112, 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.190, 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.156 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.097 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:21 2026", 2026-02-13T06:39:23.708 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.140, 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.137, 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.188 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.109, 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.090, 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.090 2026-02-13T06:39:23.709 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.181, 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.217, 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 1.246 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.126 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:22 2026", 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:23.710 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.119, 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.139, 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.178 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.103, 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.097, 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.097 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.129, 2026-02-13T06:39:23.711 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.206, 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.510 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.116 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:22 2026", 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.123, 2026-02-13T06:39:23.712 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.142, 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.172 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.107, 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.102, 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.102 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.155, 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.206, 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.492 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.713 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.107 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:21 2026", 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.115, 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.148, 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.154 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.714 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.105, 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.098, 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.098 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.151, 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.308, 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.327 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.106 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:21 2026", 2026-02-13T06:39:23.715 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.108, 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.142, 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.151 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.095, 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.080, 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.080 2026-02-13T06:39:23.716 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.138, 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.296, 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.442 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.100 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:21 2026", 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:39:23.717 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.109, 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.133, 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.144 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.098, 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.085, 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.085 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.116, 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.280, 2026-02-13T06:39:23.718 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.307 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.116 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:19 2026", 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.074, 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.086, 2026-02-13T06:39:23.719 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.133 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.067, 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.061, 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.061 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.082, 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.144, 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.445 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.081 2026-02-13T06:39:23.720 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:22 2026", 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.104, 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.111, 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.130 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.721 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.100, 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.082, 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.079 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.113, 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.193, 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.337 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.103 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:22 2026", 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.722 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.094, 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.103, 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.126 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.086, 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.080, 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.080 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.723 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.104, 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.181, 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.218 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.095 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:21 2026", 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:39:23.724 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.092, 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.114, 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.118 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.088, 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.081, 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.080 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.104, 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.271, 2026-02-13T06:39:23.725 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.271 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.092 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:39:19 2026", 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.085, 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.092, 2026-02-13T06:39:23.726 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.116 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.075, 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.074, 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.074 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.092, 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.158, 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.203 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.089 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T06:39:23.727 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T06:39:23.728 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:39:23.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:23.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:39:23.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: test 12 = 12 2026-02-13T06:39:23.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:39:23.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: jq .threshold 2026-02-13T06:39:23.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:39:23.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:81: TEST_network_ping_test1: sleep 50 2026-02-13T06:40:13.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:82: TEST_network_ping_test1: flush_pg_stats 2026-02-13T06:40:13.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:40:13.718 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:40:13.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:40:13.943 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T06:40:13.943 INFO:tasks.workunit.client.0.trial040.stderr:2' 2026-02-13T06:40:13.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:40:13.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:40:13.943 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:40:14.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836508 2026-02-13T06:40:14.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836508 2026-02-13T06:40:14.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836508' 2026-02-13T06:40:14.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:40:14.010 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:40:14.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705691 2026-02-13T06:40:14.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705691 2026-02-13T06:40:14.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836508 1-38654705691' 2026-02-13T06:40:14.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:40:14.075 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T06:40:14.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574875 2026-02-13T06:40:14.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574875 2026-02-13T06:40:14.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574875' 2026-02-13T06:40:14.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:40:14.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836508 2026-02-13T06:40:14.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:40:14.143 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:40:14.144 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836508 2026-02-13T06:40:14.144 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:40:14.145 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836508 2026-02-13T06:40:14.145 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-13T06:40:14.146 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836508 2026-02-13T06:40:14.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:40:14.375 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836506 -lt 21474836508 2026-02-13T06:40:14.375 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:40:15.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:40:15.377 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:40:15.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-13T06:40:15.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:40:15.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705691 2026-02-13T06:40:15.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:40:15.605 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:40:15.606 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705691 2026-02-13T06:40:15.606 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:40:15.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705691 2026-02-13T06:40:15.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705691' 2026-02-13T06:40:15.607 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 38654705691 2026-02-13T06:40:15.608 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:40:15.833 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705692 -lt 38654705691 2026-02-13T06:40:15.833 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:40:15.833 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574875 2026-02-13T06:40:15.833 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:40:15.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T06:40:15.835 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574875 2026-02-13T06:40:15.835 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:40:15.836 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574875 2026-02-13T06:40:15.837 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 55834574875 2026-02-13T06:40:15.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574875' 2026-02-13T06:40:15.837 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T06:40:16.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574875 -lt 55834574875 2026-02-13T06:40:16.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:40:16.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T06:40:16.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:40:16.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:40:16.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:40:16.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:16.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:16.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-osd.0.asok 2026-02-13T06:40:16.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:40:16.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-osd.0.asok dump_osd_network 2026-02-13T06:40:16.107 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:40:16.107 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 1000, 2026-02-13T06:40:16.107 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [] 2026-02-13T06:40:16.107 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:40:16.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:40:16.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:40:16.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: jq .threshold 2026-02-13T06:40:16.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T06:40:16.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:40:16.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T06:40:16.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T06:40:16.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T06:40:16.119 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:40:16.119 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:16.119 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:16.119 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mgr.x.asok 2026-02-13T06:40:16.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:40:16.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-mgr.x.asok dump_osd_network 2026-02-13T06:40:16.159 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:40:16.160 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 1000, 2026-02-13T06:40:16.160 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [] 2026-02-13T06:40:16.160 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:40:16.165 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.165 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:40:16.167 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:40:16.168 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.168 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: jq .threshold 2026-02-13T06:40:16.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T06:40:16.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:40:16.171 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T06:40:16.171 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:40:16.171 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:40:16.171 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:40:16.172 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:16.172 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:16.172 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-osd.0.asok 2026-02-13T06:40:16.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:40:16.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-osd.0.asok dump_osd_network 0 2026-02-13T06:40:16.216 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:40:16.216 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 0, 2026-02-13T06:40:16.216 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [ 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:16 2026", 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.151, 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.112, 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.106 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.217 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.222, 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.222, 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.222 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.222, 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.222, 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.222 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.222 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.218 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:16 2026", 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.144, 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.116, 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.126 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.200, 2026-02-13T06:40:16.219 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.200, 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.217 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.200, 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.200, 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.217 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.200 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:12 2026", 2026-02-13T06:40:16.220 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.122, 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.129, 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.137 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.133, 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.239, 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.239 2026-02-13T06:40:16.221 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.133, 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.239, 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.239 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.190 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:12 2026", 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:40:16.222 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.107, 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.113, 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.134 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.123, 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.168, 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.262 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.123, 2026-02-13T06:40:16.223 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.168, 2026-02-13T06:40:16.224 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.262 2026-02-13T06:40:16.224 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.224 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.212 2026-02-13T06:40:16.224 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T06:40:16.224 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T06:40:16.224 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:40:16.225 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.225 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:40:16.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: test 4 = 4 2026-02-13T06:40:16.225 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.225 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: jq .threshold 2026-02-13T06:40:16.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:40:16.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:40:16.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T06:40:16.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T06:40:16.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T06:40:16.228 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:40:16.228 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:16.228 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:16.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mgr.x.asok 2026-02-13T06:40:16.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:40:16.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-mgr.x.asok dump_osd_network 0 2026-02-13T06:40:16.267 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:40:16.267 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 0, 2026-02-13T06:40:16.267 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [ 2026-02-13T06:40:16.267 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:11 2026", 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.136, 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.115, 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.137 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.268 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.111, 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.070, 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.070 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.167, 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.167, 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.308 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.151 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:12 2026", 2026-02-13T06:40:16.269 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.122, 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.129, 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.137 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.116, 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.085, 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.085 2026-02-13T06:40:16.270 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.133, 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.239, 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.239 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.123 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:12 2026", 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:40:16.271 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.107, 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.113, 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.134 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.096, 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.085, 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.085 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.272 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.123, 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.168, 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.262 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.126 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:11 2026", 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.273 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.120, 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.121, 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.130 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.094, 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.094, 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.078 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.149, 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.163, 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.296 2026-02-13T06:40:16.274 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.138 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:11 2026", 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.087, 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.113, 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.127 2026-02-13T06:40:16.275 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.063, 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.063, 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.063 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.119, 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.170, 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.217 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.107 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.276 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:11 2026", 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.102, 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.103, 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.125 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.088, 2026-02-13T06:40:16.277 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.080, 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.080 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.109, 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.138, 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.280 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.117 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:11 2026", 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.278 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 2, 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.092, 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.097, 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.111 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.087, 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.081, 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.081 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.279 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.101, 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.131, 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.289 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.127 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:12 2026", 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:40:16.280 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.103, 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.101, 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.106 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.093, 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.080, 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.076 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.123, 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.138, 2026-02-13T06:40:16.281 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.193 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.122 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:12 2026", 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 2, 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.090, 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.091, 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.106 2026-02-13T06:40:16.282 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.083, 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.063, 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.063 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.107, 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.135, 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.182 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.160 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.283 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:11 2026", 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 0, 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 1, 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.089, 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.104, 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.104 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.079, 2026-02-13T06:40:16.284 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.077, 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.076 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.101, 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.222, 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.222 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.099 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:14 2026", 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.285 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "back", 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.097, 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.086, 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.091 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.070, 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.062, 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.061 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.286 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.134, 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.134, 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.163 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.134 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "last update": "Fri Feb 13 06:40:14 2026", 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "stale": false, 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "from osd": 1, 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "to osd": 0, 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "interface": "front", 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "average": { 2026-02-13T06:40:16.287 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.084, 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.088, 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.094 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "min": { 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.070, 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.070, 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.070 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "max": { 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "1min": 0.106, 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "5min": 0.113, 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: "15min": 0.158 2026-02-13T06:40:16.288 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T06:40:16.289 INFO:tasks.workunit.client.0.trial040.stdout: "last": 0.106 2026-02-13T06:40:16.289 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T06:40:16.289 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T06:40:16.289 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:40:16.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T06:40:16.291 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: test 12 = 12 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: jq .threshold 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: test 0 = 0 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mgr.x.asok 2026-02-13T06:40:16.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:40:16.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-mgr.x.asok dump_osd_network 99 2026-02-13T06:40:16.318 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:40:16.319 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 99, 2026-02-13T06:40:16.319 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [] 2026-02-13T06:40:16.319 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:40:16.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: jq .threshold 2026-02-13T06:40:16.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: test 99 = 99 2026-02-13T06:40:16.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T06:40:16.327 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T06:40:16.327 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:40:16.327 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:40:16.327 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:40:16.327 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:16.327 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:16.327 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-osd.0.asok 2026-02-13T06:40:16.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T06:40:16.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.33109/ceph-osd.0.asok dump_osd_network 98 2026-02-13T06:40:16.372 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:40:16.372 INFO:tasks.workunit.client.0.trial040.stdout: "threshold": 98, 2026-02-13T06:40:16.372 INFO:tasks.workunit.client.0.trial040.stdout: "entries": [] 2026-02-13T06:40:16.372 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:40:16.377 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T06:40:16.377 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: jq .threshold 2026-02-13T06:40:16.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: test 98 = 98 2026-02-13T06:40:16.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:105: TEST_network_ping_test1: rm -f td/network-ping/json 2026-02-13T06:40:16.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-13T06:40:16.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T06:40:16.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:40:16.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T06:40:16.380 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:40:16.380 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:40:16.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:40:16.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:40:16.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:40:16.488 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:40:16.488 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:40:16.488 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:40:16.489 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:40:16.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:40:16.490 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:40:16.490 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:40:16.491 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:40:16.491 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:40:16.491 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:40:16.491 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:40:16.492 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:40:16.493 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:40:16.493 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T06:40:17.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:40:17.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:17.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:17.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.33109 2026-02-13T06:40:17.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:40:17.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:40:17.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-13T06:40:17.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:40:17.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:40:17.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:40:17.114 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:40:17.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:40:17.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:40:17.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:40:17.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:40:17.116 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:40:17.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:40:17.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:40:17.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:40:17.117 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:40:17.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:40:17.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:40:17.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T06:40:17.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:40:17.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:17.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:17.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.33109 2026-02-13T06:40:17.121 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:40:17.121 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:40:17.121 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-13T06:40:17.122 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:40:17.122 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:17.122 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:17.122 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.33109 2026-02-13T06:40:17.123 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test2 td/network-ping 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:111: TEST_network_ping_test2: local dir=td/network-ping 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:113: TEST_network_ping_test2: export CEPH_ARGS 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: export 'EXTRA_OPTS= --mon_warn_on_slow_ping_time=0.001' 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: EXTRA_OPTS=' --mon_warn_on_slow_ping_time=0.001' 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:115: TEST_network_ping_test2: run_mon td/network-ping a 2026-02-13T06:40:17.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-13T06:40:17.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:40:17.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:40:17.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:40:17.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-13T06:40:17.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-13T06:40:17.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:40:17.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:40:17.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:40:17.146 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:40:17.146 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:17.146 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:17.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:40:17.147 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --mon_warn_on_slow_ping_time=0.001 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:40:17.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:40:17.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:40:17.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:40:17.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:40:17.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:40:17.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:40:17.175 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:40:17.175 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:40:17.176 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:40:17.176 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:40:17.176 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:17.176 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:17.176 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mon.a.asok 2026-02-13T06:40:17.176 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:40:17.176 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.33109/ceph-mon.a.asok config get fsid 2026-02-13T06:40:17.219 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:40:17.219 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:40:17.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:40:17.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:40:17.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:40:17.220 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:40:17.220 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:40:17.220 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:40:17.221 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:40:17.221 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:17.221 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:17.221 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.33109/ceph-mon.a.asok 2026-02-13T06:40:17.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:40:17.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.33109/ceph-mon.a.asok config get mon_host 2026-02-13T06:40:17.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:116: TEST_network_ping_test2: run_mgr td/network-ping x 2026-02-13T06:40:17.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-13T06:40:17.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:40:17.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:40:17.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:40:17.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-13T06:40:17.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:40:17.366 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:40:17.366 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:40:17.366 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:40:17.367 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:40:17.367 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:17.367 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:17.367 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:40:17.367 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:40:17.368 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --mon_warn_on_slow_ping_time=0.001 --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:40:17.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:117: TEST_network_ping_test2: run_osd td/network-ping 0 2026-02-13T06:40:17.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T06:40:17.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:40:17.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:40:17.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T06:40:17.389 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T06:40:17.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:40:17.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:40:17.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:40:17.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:40:17.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:40:17.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:40:17.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-13T06:40:17.391 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:40:17.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d48e22f2-9349-4d41-87db-69a8a1bc4d28 2026-02-13T06:40:17.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d48e22f2-9349-4d41-87db-69a8a1bc4d28' 2026-02-13T06:40:17.392 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 d48e22f2-9349-4d41-87db-69a8a1bc4d28 2026-02-13T06:40:17.392 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:40:17.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBRx45pqLDLFxAAOlJPDNHJoIFjGyNJFjuBUg== 2026-02-13T06:40:17.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBRx45pqLDLFxAAOlJPDNHJoIFjGyNJFjuBUg=="}' 2026-02-13T06:40:17.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d48e22f2-9349-4d41-87db-69a8a1bc4d28 -i td/network-ping/0/new.json 2026-02-13T06:40:17.504 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:40:17.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-13T06:40:17.511 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBRx45pqLDLFxAAOlJPDNHJoIFjGyNJFjuBUg== --osd-uuid d48e22f2-9349-4d41-87db-69a8a1bc4d28 2026-02-13T06:40:17.531 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:17.528+0000 7f7a1000d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:17.532 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:17.530+0000 7f7a1000d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:17.533 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:17.531+0000 7f7a1000d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:17.534 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:17.531+0000 7f7a1000d900 -1 bdev(0x56178f770c00 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:40:17.534 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:17.531+0000 7f7a1000d900 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-13T06:40:17.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-13T06:40:17.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:40:17.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:40:17.978 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:40:17.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:40:18.086 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:40:18.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:40:18.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:40:18.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:40:18.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:40:18.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:40:18.104 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:18.102+0000 7f0de1af3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:18.105 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:18.103+0000 7f0de1af3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:18.106 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:18.104+0000 7f0de1af3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:18.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:40:18.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:40:18.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:40:18.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:40:18.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:40:18.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:40:18.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:40:18.186 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:40:18.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:40:18.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:40:18.284 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:40:18.304 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:18.301+0000 7f0de1af3900 -1 Falling back to public interface 2026-02-13T06:40:18.634 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:18.631+0000 7f0de1af3900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:40:19.285 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:40:19.285 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:40:19.286 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:40:19.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:40:19.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:40:19.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:40:19.388 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1632149542,v1:127.0.0.1:6801/1632149542] [v2:127.0.0.1:6802/1632149542,v1:127.0.0.1:6803/1632149542] exists,up d48e22f2-9349-4d41-87db-69a8a1bc4d28 2026-02-13T06:40:19.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:40:19.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:40:19.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:118: TEST_network_ping_test2: run_osd td/network-ping 1 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:40:19.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:40:19.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-13T06:40:19.395 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:40:19.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8363b9c5-3441-4893-b583-aa02143e6732 2026-02-13T06:40:19.396 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 8363b9c5-3441-4893-b583-aa02143e6732 2026-02-13T06:40:19.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 8363b9c5-3441-4893-b583-aa02143e6732' 2026-02-13T06:40:19.396 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:40:19.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBTx45pKqj+FxAAFCWlVgN4JKSBohz09HNZiQ== 2026-02-13T06:40:19.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBTx45pKqj+FxAAFCWlVgN4JKSBohz09HNZiQ=="}' 2026-02-13T06:40:19.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8363b9c5-3441-4893-b583-aa02143e6732 -i td/network-ping/1/new.json 2026-02-13T06:40:19.506 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:40:19.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-13T06:40:19.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBTx45pKqj+FxAAFCWlVgN4JKSBohz09HNZiQ== --osd-uuid 8363b9c5-3441-4893-b583-aa02143e6732 2026-02-13T06:40:19.532 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:19.530+0000 7f95688af900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:19.534 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:19.531+0000 7f95688af900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:19.535 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:19.532+0000 7f95688af900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:19.535 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:19.533+0000 7f95688af900 -1 bdev(0x563356a80c00 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-13T06:40:19.535 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:19.533+0000 7f95688af900 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-13T06:40:19.875 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-13T06:40:19.875 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:40:19.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T06:40:19.876 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T06:40:19.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:40:20.189 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T06:40:20.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T06:40:20.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:40:20.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:40:20.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:40:20.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:40:20.208 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:20.205+0000 7f1fc06af900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:20.209 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:20.207+0000 7f1fc06af900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:20.210 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:20.208+0000 7f1fc06af900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:20.342 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:20.340+0000 7f1fc06af900 -1 Falling back to public interface 2026-02-13T06:40:20.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T06:40:20.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:40:20.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T06:40:20.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:40:20.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:40:20.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:40:20.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:40:20.420 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:40:20.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:40:20.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:40:20.637 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:20.634+0000 7f1fc06af900 -1 osd.1 0 log_to_monitors true 2026-02-13T06:40:20.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:40:21.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:40:21.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:40:21.664 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:40:21.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:40:21.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:40:21.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:40:21.888 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/933519404,v1:127.0.0.1:6811/933519404] [v2:127.0.0.1:6812/933519404,v1:127.0.0.1:6813/933519404] exists,up 8363b9c5-3441-4893-b583-aa02143e6732 2026-02-13T06:40:21.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:40:21.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:40:21.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:119: TEST_network_ping_test2: run_osd td/network-ping 2 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:40:21.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' 2026-02-13T06:40:21.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-13T06:40:21.895 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:40:21.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=daa8c60f-2695-4ddc-aa5c-1002459ca705 2026-02-13T06:40:21.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 daa8c60f-2695-4ddc-aa5c-1002459ca705' 2026-02-13T06:40:21.896 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 daa8c60f-2695-4ddc-aa5c-1002459ca705 2026-02-13T06:40:21.896 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:40:21.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBVx45pWPvINRAAsOgQF0W9s30Q8YnczygT+w== 2026-02-13T06:40:21.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBVx45pWPvINRAAsOgQF0W9s30Q8YnczygT+w=="}' 2026-02-13T06:40:21.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new daa8c60f-2695-4ddc-aa5c-1002459ca705 -i td/network-ping/2/new.json 2026-02-13T06:40:22.137 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T06:40:22.145 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-13T06:40:22.146 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBVx45pWPvINRAAsOgQF0W9s30Q8YnczygT+w== --osd-uuid daa8c60f-2695-4ddc-aa5c-1002459ca705 2026-02-13T06:40:22.164 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:22.162+0000 7f94276a1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:22.165 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:22.163+0000 7f94276a1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:22.166 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:22.164+0000 7f94276a1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:22.167 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:22.164+0000 7f94276a1900 -1 bdev(0x5640e653ec00 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-13T06:40:22.167 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:22.164+0000 7f94276a1900 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-13T06:40:22.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-13T06:40:22.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:40:22.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T06:40:22.481 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T06:40:22.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:40:22.812 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T06:40:22.812 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T06:40:22.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=be61887b-344b-4e33-a061-10a52d075faa --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.33109/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:40:22.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:40:22.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:40:22.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:40:22.830 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:22.827+0000 7fd6dfde0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:22.831 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:22.828+0000 7fd6dfde0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:22.832 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:22.829+0000 7fd6dfde0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:40:22.957 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:22.954+0000 7fd6dfde0900 -1 Falling back to public interface 2026-02-13T06:40:23.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T06:40:23.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:40:23.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T06:40:23.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:40:23.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:40:23.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:40:23.038 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:40:23.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:40:23.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:40:23.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T06:40:23.237 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:40:23.235+0000 7fd6dfde0900 -1 osd.2 0 log_to_monitors true 2026-02-13T06:40:23.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:40:24.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:40:24.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:40:24.269 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:40:24.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:40:24.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:40:24.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T06:40:24.497 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2103591817,v1:127.0.0.1:6819/2103591817] [v2:127.0.0.1:6820/2103591817,v1:127.0.0.1:6821/2103591817] exists,up daa8c60f-2695-4ddc-aa5c-1002459ca705 2026-02-13T06:40:24.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:40:24.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:40:24.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:40:24.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:121: TEST_network_ping_test2: sleep 5 2026-02-13T06:40:29.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:122: TEST_network_ping_test2: ceph osd crush add-bucket dc1 datacenter 2026-02-13T06:40:29.773 INFO:tasks.workunit.client.0.trial040.stderr:bucket 'dc1' already exists 2026-02-13T06:40:29.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:123: TEST_network_ping_test2: ceph osd crush add-bucket dc2 datacenter 2026-02-13T06:40:30.052 INFO:tasks.workunit.client.0.trial040.stderr:bucket 'dc2' already exists 2026-02-13T06:40:30.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:124: TEST_network_ping_test2: ceph osd crush add-bucket dc3 datacenter 2026-02-13T06:40:30.332 INFO:tasks.workunit.client.0.trial040.stderr:bucket 'dc3' already exists 2026-02-13T06:40:30.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:125: TEST_network_ping_test2: ceph osd crush add-bucket rack1 rack 2026-02-13T06:40:30.641 INFO:tasks.workunit.client.0.trial040.stderr:bucket 'rack1' already exists 2026-02-13T06:40:30.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:126: TEST_network_ping_test2: ceph osd crush add-bucket rack2 rack 2026-02-13T06:40:30.923 INFO:tasks.workunit.client.0.trial040.stderr:bucket 'rack2' already exists 2026-02-13T06:40:30.931 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:127: TEST_network_ping_test2: ceph osd crush add-bucket rack3 rack 2026-02-13T06:40:31.204 INFO:tasks.workunit.client.0.trial040.stderr:bucket 'rack3' already exists 2026-02-13T06:40:31.211 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:128: TEST_network_ping_test2: ceph osd crush add-bucket host1 host 2026-02-13T06:40:31.483 INFO:tasks.workunit.client.0.trial040.stderr:bucket 'host1' already exists 2026-02-13T06:40:31.491 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:129: TEST_network_ping_test2: ceph osd crush add-bucket host2 host 2026-02-13T06:40:31.765 INFO:tasks.workunit.client.0.trial040.stderr:bucket 'host2' already exists 2026-02-13T06:40:31.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:130: TEST_network_ping_test2: ceph osd crush add-bucket host3 host 2026-02-13T06:40:32.043 INFO:tasks.workunit.client.0.trial040.stderr:bucket 'host3' already exists 2026-02-13T06:40:32.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:131: TEST_network_ping_test2: ceph osd crush move dc1 root=default 2026-02-13T06:40:32.324 INFO:tasks.workunit.client.0.trial040.stderr:no need to move item id -5 name 'dc1' to location {root=default} in crush map 2026-02-13T06:40:32.332 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:132: TEST_network_ping_test2: ceph osd crush move dc2 root=default 2026-02-13T06:40:32.627 INFO:tasks.workunit.client.0.trial040.stderr:no need to move item id -6 name 'dc2' to location {root=default} in crush map 2026-02-13T06:40:32.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:133: TEST_network_ping_test2: ceph osd crush move dc3 root=default 2026-02-13T06:40:32.906 INFO:tasks.workunit.client.0.trial040.stderr:no need to move item id -7 name 'dc3' to location {root=default} in crush map 2026-02-13T06:40:32.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:134: TEST_network_ping_test2: ceph osd crush move rack1 datacenter=dc1 2026-02-13T06:40:33.202 INFO:tasks.workunit.client.0.trial040.stderr:no need to move item id -8 name 'rack1' to location {datacenter=dc1} in crush map 2026-02-13T06:40:33.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:135: TEST_network_ping_test2: ceph osd crush move rack2 datacenter=dc2 2026-02-13T06:40:33.480 INFO:tasks.workunit.client.0.trial040.stderr:no need to move item id -9 name 'rack2' to location {datacenter=dc2} in crush map 2026-02-13T06:40:33.488 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:136: TEST_network_ping_test2: ceph osd crush move rack3 datacenter=dc3 2026-02-13T06:40:33.760 INFO:tasks.workunit.client.0.trial040.stderr:no need to move item id -10 name 'rack3' to location {datacenter=dc3} in crush map 2026-02-13T06:40:33.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:137: TEST_network_ping_test2: ceph osd crush move host1 rack=rack1 2026-02-13T06:40:34.049 INFO:tasks.workunit.client.0.trial040.stderr:no need to move item id -11 name 'host1' to location {rack=rack1} in crush map 2026-02-13T06:40:34.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:138: TEST_network_ping_test2: ceph osd crush move host2 rack=rack2 2026-02-13T06:40:34.334 INFO:tasks.workunit.client.0.trial040.stderr:no need to move item id -12 name 'host2' to location {rack=rack2} in crush map 2026-02-13T06:40:34.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:139: TEST_network_ping_test2: ceph osd crush move host3 rack=rack3 2026-02-13T06:40:34.637 INFO:tasks.workunit.client.0.trial040.stderr:no need to move item id -13 name 'host3' to location {rack=rack3} in crush map 2026-02-13T06:40:34.645 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:140: TEST_network_ping_test2: ceph osd crush set osd.0 1.0 host=host1 2026-02-13T06:40:34.917 INFO:tasks.workunit.client.0.trial040.stderr:set item id 0 name 'osd.0' weight 1 at location {host=host1}: no change 2026-02-13T06:40:34.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:141: TEST_network_ping_test2: ceph osd crush set osd.1 1.0 host=host2 2026-02-13T06:40:35.194 INFO:tasks.workunit.client.0.trial040.stderr:set item id 1 name 'osd.1' weight 1 at location {host=host2}: no change 2026-02-13T06:40:35.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:142: TEST_network_ping_test2: ceph osd crush set osd.2 1.0 host=host3 2026-02-13T06:40:35.477 INFO:tasks.workunit.client.0.trial040.stderr:set item id 2 name 'osd.2' weight 1 at location {host=host3}: no change 2026-02-13T06:40:35.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:143: TEST_network_ping_test2: ceph osd crush rule create-simple myrule default host firstn 2026-02-13T06:40:35.757 INFO:tasks.workunit.client.0.trial040.stderr:rule myrule already exists 2026-02-13T06:40:35.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:145: TEST_network_ping_test2: create_pool foo 16 16 replicated myrule 2026-02-13T06:40:35.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 16 replicated myrule 2026-02-13T06:40:36.041 INFO:tasks.workunit.client.0.trial040.stderr:pool 'foo' already exists 2026-02-13T06:40:36.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:40:37.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:148: TEST_network_ping_test2: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-13T06:40:37.074 INFO:tasks.workunit.client.0.trial040.stdout:hints = 1 2026-02-13T06:40:37.074 INFO:tasks.workunit.client.0.trial040.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-13T06:40:37.074 INFO:tasks.workunit.client.0.trial040.stdout:Object prefix: benchmark_data_trial040_43325 2026-02-13T06:40:37.074 INFO:tasks.workunit.client.0.trial040.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-13T06:40:37.074 INFO:tasks.workunit.client.0.trial040.stdout: 0 0 0 0 0 0 - 0 2026-02-13T06:40:38.074 INFO:tasks.workunit.client.0.trial040.stdout: 1 16 11628 11612 45.356 45.3594 0.00164577 0.00137588 2026-02-13T06:40:39.074 INFO:tasks.workunit.client.0.trial040.stdout: 2 16 23447 23431 45.7602 46.168 0.00133512 0.00136243 2026-02-13T06:40:40.074 INFO:tasks.workunit.client.0.trial040.stdout: 3 16 35758 35742 46.5356 48.0898 0.000904771 0.00134144 2026-02-13T06:40:41.074 INFO:tasks.workunit.client.0.trial040.stdout: 4 16 47707 47691 46.5698 46.6758 0.000758058 0.00133992 2026-02-13T06:40:42.074 INFO:tasks.workunit.client.0.trial040.stdout: 5 16 59531 59515 46.4927 46.1875 0.000791652 0.00134287 2026-02-13T06:40:43.074 INFO:tasks.workunit.client.0.trial040.stdout: 6 16 71534 71518 46.5571 46.8867 0.000677669 0.00134098 2026-02-13T06:40:44.074 INFO:tasks.workunit.client.0.trial040.stdout: 7 16 83350 83334 46.4993 46.1562 0.000964302 0.00134268 2026-02-13T06:40:45.074 INFO:tasks.workunit.client.0.trial040.stdout: 8 16 93834 93818 45.8057 40.9531 0.00137666 0.00136302 2026-02-13T06:40:46.074 INFO:tasks.workunit.client.0.trial040.stdout: 9 16 104422 104406 45.3113 41.3594 0.0011795 0.00137794 2026-02-13T06:40:47.074 INFO:tasks.workunit.client.0.trial040.stdout: 10 8 114694 114686 44.7955 40.1562 0.00137699 0.00139382 2026-02-13T06:40:48.075 INFO:tasks.workunit.client.0.trial040.stdout:Total time run: 10.0012 2026-02-13T06:40:48.075 INFO:tasks.workunit.client.0.trial040.stdout:Total writes made: 114694 2026-02-13T06:40:48.075 INFO:tasks.workunit.client.0.trial040.stdout:Write size: 4096 2026-02-13T06:40:48.075 INFO:tasks.workunit.client.0.trial040.stdout:Object size: 4096 2026-02-13T06:40:48.075 INFO:tasks.workunit.client.0.trial040.stdout:Bandwidth (MB/sec): 44.7968 2026-02-13T06:40:48.075 INFO:tasks.workunit.client.0.trial040.stdout:Stddev Bandwidth: 2.84499 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Max bandwidth (MB/sec): 48.0898 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Min bandwidth (MB/sec): 40.1562 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Average IOPS: 11467 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Stddev IOPS: 728.317 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Max IOPS: 12311 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Min IOPS: 10280 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Average Latency(s): 0.00139381 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Stddev Latency(s): 0.000697335 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Max latency(s): 0.0144375 2026-02-13T06:40:48.076 INFO:tasks.workunit.client.0.trial040.stdout:Min latency(s): 0.000442091 2026-02-13T06:40:48.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:151: TEST_network_ping_test2: sleep 10 2026-02-13T06:40:58.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:152: TEST_network_ping_test2: flush_pg_stats 2026-02-13T06:40:58.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:40:58.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:40:58.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:40:58.311 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T06:40:58.311 INFO:tasks.workunit.client.0.trial040.stderr:2' 2026-02-13T06:40:58.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:40:58.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:40:58.311 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:40:58.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-13T06:40:58.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-13T06:40:58.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490' 2026-02-13T06:40:58.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:40:58.377 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:40:58.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-13T06:40:58.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-13T06:40:58.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490 1-42949672970' 2026-02-13T06:40:58.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:40:58.441 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T06:40:58.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-13T06:40:58.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-13T06:40:58.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490 1-42949672970 2-60129542153' 2026-02-13T06:40:58.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:40:58.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-13T06:40:58.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:40:58.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:40:58.509 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836490 2026-02-13T06:40:58.509 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:40:58.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836490 2026-02-13T06:40:58.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-13T06:40:58.511 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836490 2026-02-13T06:40:58.511 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:40:58.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T06:40:58.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:40:59.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:40:59.735 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:40:59.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-13T06:40:59.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:40:59.961 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-13T06:40:59.962 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:40:59.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:40:59.963 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672970 2026-02-13T06:40:59.963 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:40:59.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672970 2026-02-13T06:40:59.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-13T06:40:59.964 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672970 2026-02-13T06:40:59.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:41:00.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-13T06:41:00.192 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:00.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-13T06:41:00.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:00.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T06:41:00.193 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542153 2026-02-13T06:41:00.194 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:00.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542153 2026-02-13T06:41:00.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-13T06:41:00.195 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 60129542153 2026-02-13T06:41:00.195 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T06:41:00.423 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-13T06:41:00.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: ceph health 2026-02-13T06:41:00.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: tee td/network-ping/health 2026-02-13T06:41:00.718 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.406ms); Slow OSD heartbeats on front (longest 0.461ms) 2026-02-13T06:41:00.725 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:155: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats' td/network-ping/health 2026-02-13T06:41:00.727 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: ceph health detail 2026-02-13T06:41:00.727 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: tee td/network-ping/health 2026-02-13T06:41:01.029 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.406ms); Slow OSD heartbeats on front (longest 0.461ms) 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout:[WRN] OSD_SLOW_PING_TIME_BACK: Slow OSD heartbeats on back (longest 0.406ms) 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.406 msec 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.250 msec 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.202 msec 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.193 msec 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.184 msec 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.129 msec possibly improving 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout:[WRN] OSD_SLOW_PING_TIME_FRONT: Slow OSD heartbeats on front (longest 0.461ms) 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.461 msec possibly improving 2026-02-13T06:41:01.030 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.446 msec 2026-02-13T06:41:01.031 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.275 msec 2026-02-13T06:41:01.031 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.241 msec 2026-02-13T06:41:01.031 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.223 msec 2026-02-13T06:41:01.031 INFO:tasks.workunit.client.0.trial040.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.151 msec 2026-02-13T06:41:01.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:158: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_BACK td/network-ping/health 2026-02-13T06:41:01.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:159: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_FRONT td/network-ping/health 2026-02-13T06:41:01.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:160: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats on front from osd[.][0-2] [[]dc[1-3],rack[1-3][]] to osd[.][0-2] [[]dc[1-3],rack[1-3][]]' td/network-ping/health 2026-02-13T06:41:01.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:162: TEST_network_ping_test2: rm -f td/network-ping/health 2026-02-13T06:41:01.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-13T06:41:01.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T06:41:01.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:01.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T06:41:01.041 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:01.041 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:01.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:01.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:01.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:01.148 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:01.148 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:01.149 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:01.149 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:01.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:01.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:01.150 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:01.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:01.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:01.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:01.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:01.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:01.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:01.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T06:41:01.890 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:01.890 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:01.890 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:41:01.890 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.33109 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/network-ping 0 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:01.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:01.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:01.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:01.893 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:01.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:01.894 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:01.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:01.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:01.896 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:01.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:01.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:01.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:01.897 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:01.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:01.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T06:41:01.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T06:41:01.900 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:01.900 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:01.900 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.33109 2026-02-13T06:41:01.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.33109 2026-02-13T06:41:01.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:01.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:01.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-13T06:41:01.902 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T06:41:01.903 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T06:41:01.967 INFO:tasks.workunit:Running workunit misc/ok-to-stop.sh... 2026-02-13T06:41:01.967 DEBUG:teuthology.orchestra.run.trial040:workunit test misc/ok-to-stop.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh 2026-02-13T06:41:02.028 INFO:tasks.workunit.client.0.trial040.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T06:41:02.033 INFO:tasks.workunit.client.0.trial040.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/ok-to-stop 2026-02-13T06:41:02.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:6: run: local dir=td/ok-to-stop 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:7: run: shift 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: export CEPH_MON_A=127.0.0.1:7150 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: CEPH_MON_A=127.0.0.1:7150 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: export CEPH_MON_B=127.0.0.1:7151 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: CEPH_MON_B=127.0.0.1:7151 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: export CEPH_MON_C=127.0.0.1:7152 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: CEPH_MON_C=127.0.0.1:7152 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: export CEPH_MON_D=127.0.0.1:7153 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: CEPH_MON_D=127.0.0.1:7153 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: export CEPH_MON_E=127.0.0.1:7154 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: CEPH_MON_E=127.0.0.1:7154 2026-02-13T06:41:02.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:14: run: export CEPH_ARGS 2026-02-13T06:41:02.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: uuidgen 2026-02-13T06:41:02.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: CEPH_ARGS+='--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none ' 2026-02-13T06:41:02.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: export 'ORIG_CEPH_ARGS=--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none ' 2026-02-13T06:41:02.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: ORIG_CEPH_ARGS='--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none ' 2026-02-13T06:41:02.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: set 2026-02-13T06:41:02.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T06:41:02.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: local 'funcs=TEST_0_mds 2026-02-13T06:41:02.038 INFO:tasks.workunit.client.0.trial040.stderr:TEST_0_osd 2026-02-13T06:41:02.038 INFO:tasks.workunit.client.0.trial040.stderr:TEST_1_mon_checks 2026-02-13T06:41:02.038 INFO:tasks.workunit.client.0.trial040.stderr:TEST_2_mons_checks 2026-02-13T06:41:02.038 INFO:tasks.workunit.client.0.trial040.stderr:TEST_3_mons_checks 2026-02-13T06:41:02.038 INFO:tasks.workunit.client.0.trial040.stderr:TEST_4_mons_checks 2026-02-13T06:41:02.038 INFO:tasks.workunit.client.0.trial040.stderr:TEST_5_mons_checks' 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:02.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:02.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:02.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:02.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:02.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:02.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:02.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:02.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:02.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:02.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:02.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:02.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:02.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:02.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:02.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:02.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:41:02.043 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:02.044 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:02.044 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:02.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:41:02.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:02.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:02.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T06:41:02.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:41:02.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:02.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:02.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.43705 2026-02-13T06:41:02.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:41:02.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T06:41:02.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T06:41:02.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:41:02.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T06:41:02.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_mds td/ok-to-stop 2026-02-13T06:41:02.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:208: TEST_0_mds: local dir=td/ok-to-stop 2026-02-13T06:41:02.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:210: TEST_0_mds: CEPH_ARGS='--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T06:41:02.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:212: TEST_0_mds: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T06:41:02.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:41:02.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:41:02.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:41:02.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:41:02.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T06:41:02.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T06:41:02.068 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:41:02.068 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:02.068 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:02.068 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:02.068 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:02.069 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:02.069 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:02.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T06:41:02.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:41:02.096 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:41:02.096 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:02.097 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:02.098 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:02.098 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get fsid 2026-02-13T06:41:02.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:41:02.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:02.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:02.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:41:02.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:41:02.143 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:02.143 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:02.144 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:02.144 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:02.144 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:02.144 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:02.144 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:02.144 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:02.144 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get mon_host 2026-02-13T06:41:02.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:213: TEST_0_mds: run_mgr td/ok-to-stop x 2026-02-13T06:41:02.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-13T06:41:02.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:41:02.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:41:02.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:41:02.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-13T06:41:02.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:41:02.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:41:02.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:02.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:02.290 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:02.290 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:02.291 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:02.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:02.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:41:02.291 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:41:02.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:214: TEST_0_mds: run_osd td/ok-to-stop 0 2026-02-13T06:41:02.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T06:41:02.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:41:02.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:41:02.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:41:02.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-13T06:41:02.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T06:41:02.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:41:02.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:41:02.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:41:02.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-13T06:41:02.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-13T06:41:02.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:41:02.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:41:02.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:41:02.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-13T06:41:02.313 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:41:02.314 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=609a4c53-a718-485a-8228-473af7dca96b 2026-02-13T06:41:02.314 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 609a4c53-a718-485a-8228-473af7dca96b' 2026-02-13T06:41:02.314 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 609a4c53-a718-485a-8228-473af7dca96b 2026-02-13T06:41:02.314 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:41:02.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB+x45pr6j/EhAAxNOUE01hsW6rpjIv9V3nww== 2026-02-13T06:41:02.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB+x45pr6j/EhAAxNOUE01hsW6rpjIv9V3nww=="}' 2026-02-13T06:41:02.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 609a4c53-a718-485a-8228-473af7dca96b -i td/ok-to-stop/0/new.json 2026-02-13T06:41:02.424 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:41:02.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-13T06:41:02.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB+x45pr6j/EhAAxNOUE01hsW6rpjIv9V3nww== --osd-uuid 609a4c53-a718-485a-8228-473af7dca96b 2026-02-13T06:41:02.448 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:02.446+0000 7fc2f93a7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:02.450 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:02.447+0000 7fc2f93a7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:02.451 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:02.448+0000 7fc2f93a7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:02.451 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:02.449+0000 7fc2f93a7900 -1 bdev(0x5653b3592c00 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:41:02.452 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:02.449+0000 7fc2f93a7900 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-13T06:41:02.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-13T06:41:02.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:41:02.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:41:02.873 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:41:02.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:41:02.974 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:41:02.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:41:02.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:41:02.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:41:02.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:41:02.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:41:02.990 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:02.988+0000 7fec02812900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:02.992 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:02.989+0000 7fec02812900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:02.993 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:02.991+0000 7fec02812900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:03.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:41:03.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:41:03.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:41:03.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:41:03.074 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:41:03.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:41:03.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:03.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:41:03.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:03.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:41:03.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:41:03.216 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:03.214+0000 7fec02812900 -1 Falling back to public interface 2026-02-13T06:41:03.578 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:03.575+0000 7fec02812900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:41:04.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:41:04.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:04.177 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:41:04.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:41:04.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:04.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:41:04.275 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1455606542,v1:127.0.0.1:6801/1455606542] [v2:127.0.0.1:6802/1455606542,v1:127.0.0.1:6803/1455606542] exists,up 609a4c53-a718-485a-8228-473af7dca96b 2026-02-13T06:41:04.277 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:41:04.277 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:41:04.277 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:41:04.277 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:215: TEST_0_mds: run_mds td/ok-to-stop a 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=a 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/a 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:04.278 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:04.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id a --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-13T06:41:04.293 INFO:tasks.workunit.client.0.trial040.stdout:starting mds.a at 2026-02-13T06:41:04.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:217: TEST_0_mds: ceph osd pool create meta 1 2026-02-13T06:41:04.469 INFO:tasks.workunit.client.0.trial040.stderr:pool 'meta' already exists 2026-02-13T06:41:04.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:218: TEST_0_mds: ceph osd pool create data 1 2026-02-13T06:41:04.712 INFO:tasks.workunit.client.0.trial040.stderr:pool 'data' already exists 2026-02-13T06:41:04.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:219: TEST_0_mds: ceph fs new myfs meta data 2026-02-13T06:41:05.061 INFO:tasks.workunit.client.0.trial040.stderr:filesystem 'myfs' already exists 2026-02-13T06:41:05.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:220: TEST_0_mds: sleep 5 2026-02-13T06:41:10.070 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:222: TEST_0_mds: ceph mds ok-to-stop a 2026-02-13T06:41:10.269 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 2026-02-13T06:41:10.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:223: TEST_0_mds: ceph mds ok-to-stop a dne 2026-02-13T06:41:10.476 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 2026-02-13T06:41:10.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:224: TEST_0_mds: ceph mds ok-to-stop dne 2026-02-13T06:41:10.786 INFO:tasks.workunit.client.0.trial040.stderr:should be safe to stop [dne] 2026-02-13T06:41:10.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:226: TEST_0_mds: run_mds td/ok-to-stop b 2026-02-13T06:41:10.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-13T06:41:10.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-13T06:41:10.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=b 2026-02-13T06:41:10.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-13T06:41:10.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/b 2026-02-13T06:41:10.794 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-13T06:41:10.795 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:10.795 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:10.795 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:10.795 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:10.795 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:10.795 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:10.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id b --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-13T06:41:10.809 INFO:tasks.workunit.client.0.trial040.stdout:starting mds.b at 2026-02-13T06:41:10.812 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:227: TEST_0_mds: sleep 5 2026-02-13T06:41:15.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:229: TEST_0_mds: ceph mds ok-to-stop a 2026-02-13T06:41:16.124 INFO:tasks.workunit.client.0.trial040.stderr:should be safe to stop [a] 2026-02-13T06:41:16.131 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:230: TEST_0_mds: ceph mds ok-to-stop b 2026-02-13T06:41:16.436 INFO:tasks.workunit.client.0.trial040.stderr:should be safe to stop [b] 2026-02-13T06:41:16.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:231: TEST_0_mds: ceph mds ok-to-stop a b 2026-02-13T06:41:16.643 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 4135 2026-02-13T06:41:16.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:232: TEST_0_mds: ceph mds ok-to-stop a dne1 dne2 2026-02-13T06:41:16.953 INFO:tasks.workunit.client.0.trial040.stderr:should be safe to stop [a,dne1,dne2] 2026-02-13T06:41:16.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:233: TEST_0_mds: ceph mds ok-to-stop b dne 2026-02-13T06:41:17.265 INFO:tasks.workunit.client.0.trial040.stderr:should be safe to stop [b,dne] 2026-02-13T06:41:17.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:234: TEST_0_mds: ceph mds ok-to-stop a b dne 2026-02-13T06:41:17.475 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 4135 2026-02-13T06:41:17.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:235: TEST_0_mds: ceph mds ok-to-stop dne1 dne2 2026-02-13T06:41:17.783 INFO:tasks.workunit.client.0.trial040.stderr:should be safe to stop [dne1,dne2] 2026-02-13T06:41:17.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:237: TEST_0_mds: kill_daemons td/ok-to-stop KILL mds.a 2026-02-13T06:41:17.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:17.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:17.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:17.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:17.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:17.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:17.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:17.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:17.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:17.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:17.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:17.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:18.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:18.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T06:41:18.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:41:18.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:18.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:18.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:18.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:18.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:18.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:18.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:18.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:18.013 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:18.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:18.014 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:18.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:18.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:18.015 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:18.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:18.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:18.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:18.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:18.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:18.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:18.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:41:18.022 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:18.022 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:18.023 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:18.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:41:18.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:18.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:18.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:18.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:18.026 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:18.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:18.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:18.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:18.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:18.028 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:18.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:18.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:18.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:18.030 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:18.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:18.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:18.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:41:18.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:18.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:18.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:18.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:41:18.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:18.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:18.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T06:41:18.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:41:18.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:18.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:18.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.43705 2026-02-13T06:41:18.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:41:18.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:41:18.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:41:18.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T06:41:18.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_osd td/ok-to-stop 2026-02-13T06:41:18.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:241: TEST_0_osd: local dir=td/ok-to-stop 2026-02-13T06:41:18.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:243: TEST_0_osd: CEPH_ARGS='--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T06:41:18.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:245: TEST_0_osd: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T06:41:18.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:41:18.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:41:18.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:41:18.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:41:18.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T06:41:18.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T06:41:18.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:41:18.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:18.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:18.057 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:18.057 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:18.057 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:18.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:18.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T06:41:18.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:41:18.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:41:18.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:18.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:18.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:41:18.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:41:18.085 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:18.086 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:18.086 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:18.086 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:18.087 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:18.087 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:18.087 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:18.087 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:18.087 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get fsid 2026-02-13T06:41:18.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:41:18.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:18.134 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:18.134 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:41:18.134 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:41:18.134 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:18.134 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:18.134 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:18.135 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:18.135 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:18.135 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:18.135 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:18.135 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:18.135 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get mon_host 2026-02-13T06:41:18.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:246: TEST_0_osd: run_mgr td/ok-to-stop x 2026-02-13T06:41:18.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-13T06:41:18.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:41:18.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:41:18.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:41:18.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-13T06:41:18.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:41:18.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:41:18.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:18.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:18.280 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:18.280 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:18.280 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:18.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:18.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:41:18.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:41:18.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:247: TEST_0_osd: run_osd td/ok-to-stop 0 2026-02-13T06:41:18.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T06:41:18.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:41:18.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:41:18.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:41:18.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:18.302 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:41:18.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:41:18.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:41:18.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:41:18.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:41:18.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-13T06:41:18.304 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:41:18.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dbbcb46b-351c-420e-8926-1d6ff0710b94 2026-02-13T06:41:18.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dbbcb46b-351c-420e-8926-1d6ff0710b94' 2026-02-13T06:41:18.304 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 dbbcb46b-351c-420e-8926-1d6ff0710b94 2026-02-13T06:41:18.305 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:41:18.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCOx45pa8d2EhAAVbVA8jvPwfg3kJZd0r8T8g== 2026-02-13T06:41:18.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCOx45pa8d2EhAAVbVA8jvPwfg3kJZd0r8T8g=="}' 2026-02-13T06:41:18.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dbbcb46b-351c-420e-8926-1d6ff0710b94 -i td/ok-to-stop/0/new.json 2026-02-13T06:41:18.416 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:41:18.423 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-13T06:41:18.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCOx45pa8d2EhAAVbVA8jvPwfg3kJZd0r8T8g== --osd-uuid dbbcb46b-351c-420e-8926-1d6ff0710b94 2026-02-13T06:41:18.440 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:18.438+0000 7ff250065900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:18.442 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:18.440+0000 7ff250065900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:18.443 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:18.441+0000 7ff250065900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:18.444 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:18.441+0000 7ff250065900 -1 bdev(0x55a88f7d8c00 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:41:18.444 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:18.441+0000 7ff250065900 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-13T06:41:18.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-13T06:41:18.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:41:18.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:41:18.872 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:41:18.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:41:18.975 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:41:18.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:41:18.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:41:18.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:41:18.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:41:18.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:41:18.992 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:18.990+0000 7febdfd70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:18.994 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:18.991+0000 7febdfd70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:18.995 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:18.992+0000 7febdfd70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:19.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:41:19.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:41:19.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:41:19.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:41:19.075 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:41:19.076 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:19.076 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:41:19.076 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:41:19.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:19.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:41:19.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:41:19.246 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:19.243+0000 7febdfd70900 -1 Falling back to public interface 2026-02-13T06:41:19.564 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:19.561+0000 7febdfd70900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:41:20.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:41:20.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:20.175 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:41:20.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:41:20.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:20.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:41:20.273 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/245676583,v1:127.0.0.1:6801/245676583] [v2:127.0.0.1:6802/245676583,v1:127.0.0.1:6803/245676583] exists,up dbbcb46b-351c-420e-8926-1d6ff0710b94 2026-02-13T06:41:20.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:41:20.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:41:20.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:41:20.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:248: TEST_0_osd: run_osd td/ok-to-stop 1 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/1 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:41:20.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/1' 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/1/journal' 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:20.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:41:20.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:41:20.282 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:41:20.282 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/1 2026-02-13T06:41:20.282 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:41:20.282 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=65db56a7-6815-4eb0-804d-dd020e6f8f04 2026-02-13T06:41:20.282 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 65db56a7-6815-4eb0-804d-dd020e6f8f04' 2026-02-13T06:41:20.282 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 65db56a7-6815-4eb0-804d-dd020e6f8f04 2026-02-13T06:41:20.283 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:41:20.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCQx45pNKQXERAAtnehnpSM/gNkkYwLEeN25g== 2026-02-13T06:41:20.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCQx45pNKQXERAAtnehnpSM/gNkkYwLEeN25g=="}' 2026-02-13T06:41:20.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 65db56a7-6815-4eb0-804d-dd020e6f8f04 -i td/ok-to-stop/1/new.json 2026-02-13T06:41:20.392 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:41:20.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/1/new.json 2026-02-13T06:41:20.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCQx45pNKQXERAAtnehnpSM/gNkkYwLEeN25g== --osd-uuid 65db56a7-6815-4eb0-804d-dd020e6f8f04 2026-02-13T06:41:20.415 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:20.413+0000 7f94e8267900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:20.417 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:20.414+0000 7f94e8267900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:20.418 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:20.415+0000 7f94e8267900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:20.418 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:20.415+0000 7f94e8267900 -1 bdev(0x55baf1fb2c00 td/ok-to-stop/1/block) open stat got: (1) Operation not permitted 2026-02-13T06:41:20.418 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:20.416+0000 7f94e8267900 -1 bluestore(td/ok-to-stop/1) _read_fsid unparsable uuid 2026-02-13T06:41:20.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/1/keyring 2026-02-13T06:41:20.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:41:20.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T06:41:20.749 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T06:41:20.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:41:21.062 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T06:41:21.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T06:41:21.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:41:21.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:41:21.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:41:21.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:41:21.079 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:21.077+0000 7f6bed1c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:21.081 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:21.078+0000 7f6bed1c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:21.082 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:21.079+0000 7f6bed1c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:21.215 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:21.214+0000 7f6bed1c6900 -1 Falling back to public interface 2026-02-13T06:41:21.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T06:41:21.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:41:21.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T06:41:21.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:41:21.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:41:21.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:21.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:41:21.288 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:41:21.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:21.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:41:21.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:41:21.541 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:21.539+0000 7f6bed1c6900 -1 osd.1 0 log_to_monitors true 2026-02-13T06:41:22.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:41:22.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:22.514 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:41:22.514 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:41:22.514 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:22.514 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:41:22.741 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1797654944,v1:127.0.0.1:6811/1797654944] [v2:127.0.0.1:6812/1797654944,v1:127.0.0.1:6813/1797654944] exists,up 65db56a7-6815-4eb0-804d-dd020e6f8f04 2026-02-13T06:41:22.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:41:22.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:41:22.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:41:22.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:249: TEST_0_osd: run_osd td/ok-to-stop 2 2026-02-13T06:41:22.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T06:41:22.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:41:22.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/2 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/2' 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/2/journal' 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T06:41:22.746 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T06:41:22.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/2 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dde5444f-f7bd-4971-af4e-ead957ddb4ee 2026-02-13T06:41:22.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 dde5444f-f7bd-4971-af4e-ead957ddb4ee' 2026-02-13T06:41:22.749 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 dde5444f-f7bd-4971-af4e-ead957ddb4ee 2026-02-13T06:41:22.749 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:41:22.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCSx45p2Vz4LBAAu6r//nCHQOoXHEHqqHiimg== 2026-02-13T06:41:22.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCSx45p2Vz4LBAAu6r//nCHQOoXHEHqqHiimg=="}' 2026-02-13T06:41:22.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dde5444f-f7bd-4971-af4e-ead957ddb4ee -i td/ok-to-stop/2/new.json 2026-02-13T06:41:22.987 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T06:41:22.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/2/new.json 2026-02-13T06:41:22.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCSx45p2Vz4LBAAu6r//nCHQOoXHEHqqHiimg== --osd-uuid dde5444f-f7bd-4971-af4e-ead957ddb4ee 2026-02-13T06:41:23.012 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:23.011+0000 7f2a91021900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:23.014 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:23.012+0000 7f2a91021900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:23.015 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:23.013+0000 7f2a91021900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:23.015 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:23.014+0000 7f2a91021900 -1 bdev(0x5609aad0cc00 td/ok-to-stop/2/block) open stat got: (1) Operation not permitted 2026-02-13T06:41:23.015 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:23.014+0000 7f2a91021900 -1 bluestore(td/ok-to-stop/2) _read_fsid unparsable uuid 2026-02-13T06:41:23.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/2/keyring 2026-02-13T06:41:23.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:41:23.345 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T06:41:23.345 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T06:41:23.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:41:23.667 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T06:41:23.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T06:41:23.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:41:23.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:41:23.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:41:23.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:41:23.683 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:23.682+0000 7fe5d6556900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:23.684 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:23.683+0000 7fe5d6556900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:23.685 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:23.684+0000 7fe5d6556900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:23.828 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:23.826+0000 7fe5d6556900 -1 Falling back to public interface 2026-02-13T06:41:23.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T06:41:23.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:41:23.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T06:41:23.899 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:41:23.899 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:41:23.899 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:23.899 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:41:23.899 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:41:23.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:23.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T06:41:24.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:41:24.159 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:24.157+0000 7fe5d6556900 -1 osd.2 0 log_to_monitors true 2026-02-13T06:41:25.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:41:25.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:25.127 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:41:25.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:41:25.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:25.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T06:41:25.352 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1824586669,v1:127.0.0.1:6819/1824586669] [v2:127.0.0.1:6820/1824586669,v1:127.0.0.1:6821/1824586669] exists,up dde5444f-f7bd-4971-af4e-ead957ddb4ee 2026-02-13T06:41:25.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:41:25.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:41:25.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:41:25.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:250: TEST_0_osd: run_osd td/ok-to-stop 3 2026-02-13T06:41:25.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T06:41:25.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/3 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/3' 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/3/journal' 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:41:25.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:25.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:41:25.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/3 2026-02-13T06:41:25.361 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:41:25.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c79bf74d-a5b3-4489-8394-4f4940aac96f 2026-02-13T06:41:25.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 c79bf74d-a5b3-4489-8394-4f4940aac96f' 2026-02-13T06:41:25.361 INFO:tasks.workunit.client.0.trial040.stdout:add osd3 c79bf74d-a5b3-4489-8394-4f4940aac96f 2026-02-13T06:41:25.362 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:41:25.368 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCVx45p+n7XFRAA8uQ/UN6VKVuZGD+9mHTLNA== 2026-02-13T06:41:25.368 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCVx45p+n7XFRAA8uQ/UN6VKVuZGD+9mHTLNA=="}' 2026-02-13T06:41:25.368 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c79bf74d-a5b3-4489-8394-4f4940aac96f -i td/ok-to-stop/3/new.json 2026-02-13T06:41:25.607 INFO:tasks.workunit.client.0.trial040.stdout:3 2026-02-13T06:41:25.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/3/new.json 2026-02-13T06:41:25.616 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCVx45p+n7XFRAA8uQ/UN6VKVuZGD+9mHTLNA== --osd-uuid c79bf74d-a5b3-4489-8394-4f4940aac96f 2026-02-13T06:41:25.634 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:25.632+0000 7f8eaf1c1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:25.635 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:25.634+0000 7f8eaf1c1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:25.636 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:25.634+0000 7f8eaf1c1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:25.636 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:25.635+0000 7f8eaf1c1900 -1 bdev(0x55606b1a8c00 td/ok-to-stop/3/block) open stat got: (1) Operation not permitted 2026-02-13T06:41:25.637 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:25.635+0000 7f8eaf1c1900 -1 bluestore(td/ok-to-stop/3) _read_fsid unparsable uuid 2026-02-13T06:41:25.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/3/keyring 2026-02-13T06:41:25.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:41:25.959 INFO:tasks.workunit.client.0.trial040.stdout:adding osd3 key to auth repository 2026-02-13T06:41:25.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T06:41:25.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:41:26.274 INFO:tasks.workunit.client.0.trial040.stdout:start osd.3 2026-02-13T06:41:26.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T06:41:26.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:41:26.276 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:41:26.276 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:41:26.276 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:41:26.291 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:26.289+0000 7fb70b32d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:26.292 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:26.291+0000 7fb70b32d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:26.293 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:26.292+0000 7fb70b32d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:41:26.454 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:26.453+0000 7fb70b32d900 -1 Falling back to public interface 2026-02-13T06:41:26.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T06:41:26.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:41:26.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T06:41:26.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:41:26.503 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:41:26.503 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:26.503 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:41:26.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:41:26.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:26.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T06:41:26.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:41:26.806 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:41:26.805+0000 7fb70b32d900 -1 osd.3 0 log_to_monitors true 2026-02-13T06:41:27.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:41:27.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:41:27.737 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:41:27.738 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:41:27.738 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:41:27.738 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T06:41:27.965 INFO:tasks.workunit.client.0.trial040.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3080302581,v1:127.0.0.1:6827/3080302581] [v2:127.0.0.1:6828/3080302581,v1:127.0.0.1:6829/3080302581] exists,up c79bf74d-a5b3-4489-8394-4f4940aac96f 2026-02-13T06:41:27.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:41:27.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:41:27.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:41:27.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:252: TEST_0_osd: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-13T06:41:28.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:253: TEST_0_osd: ceph osd pool create ec erasure ec-profile 2026-02-13T06:41:28.657 INFO:tasks.workunit.client.0.trial040.stderr:pool 'ec' already exists 2026-02-13T06:41:28.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:255: TEST_0_osd: wait_for_clean 2026-02-13T06:41:28.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:41:28.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:41:28.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:41:28.667 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:41:28.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:41:28.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:41:28.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:41:28.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:41:28.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:41:28.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:41:28.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:41:28.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:41:28.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:41:28.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:41:28.754 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:41:28.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:41:28.981 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T06:41:28.981 INFO:tasks.workunit.client.0.trial040.stderr:2 2026-02-13T06:41:28.981 INFO:tasks.workunit.client.0.trial040.stderr:3' 2026-02-13T06:41:28.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:41:28.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:28.982 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:41:29.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T06:41:29.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T06:41:29.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T06:41:29.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:29.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:41:29.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-13T06:41:29.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-13T06:41:29.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672964' 2026-02-13T06:41:29.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:29.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T06:41:29.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-13T06:41:29.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-13T06:41:29.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672964 2-60129542147' 2026-02-13T06:41:29.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:29.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-13T06:41:29.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-13T06:41:29.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-13T06:41:29.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672964 2-60129542147 3-77309411332' 2026-02-13T06:41:29.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:29.239 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T06:41:29.240 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:29.240 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:41:29.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-13T06:41:29.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:29.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-13T06:41:29.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T06:41:29.242 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836484 2026-02-13T06:41:29.243 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:41:29.473 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T06:41:29.473 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:41:30.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:41:30.475 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:41:30.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-13T06:41:30.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:30.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-13T06:41:30.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:30.706 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:41:30.707 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672964 2026-02-13T06:41:30.707 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:30.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672964 2026-02-13T06:41:30.708 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672964 2026-02-13T06:41:30.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-13T06:41:30.709 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:41:30.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-13T06:41:30.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:30.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-13T06:41:30.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:30.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T06:41:30.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542147 2026-02-13T06:41:30.934 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:30.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542147 2026-02-13T06:41:30.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-13T06:41:30.935 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 60129542147 2026-02-13T06:41:30.936 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T06:41:31.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542147 2026-02-13T06:41:31.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:31.161 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411332 2026-02-13T06:41:31.161 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:31.162 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-13T06:41:31.162 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-77309411332 2026-02-13T06:41:31.162 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:31.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411332 2026-02-13T06:41:31.164 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.3 seq 77309411332 2026-02-13T06:41:31.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 77309411332' 2026-02-13T06:41:31.164 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T06:41:31.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411332 -lt 77309411332 2026-02-13T06:41:31.394 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:41:31.395 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:41:31.395 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:41:31.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-13T06:41:31.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:41:31.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:41:31.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:41:31.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:41:31.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:41:31.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:41:31.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:41:31.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-13T06:41:31.951 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:41:31.951 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:41:31.951 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:41:32.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-13T06:41:32.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:41:32.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:41:32.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:258: TEST_0_osd: ceph osd pool set ec min_size 3 2026-02-13T06:41:32.606 INFO:tasks.workunit.client.0.trial040.stderr:set pool 1 min_size to 3 2026-02-13T06:41:32.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:259: TEST_0_osd: wait_for_clean 2026-02-13T06:41:32.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:41:32.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:41:32.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:41:32.618 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:41:32.618 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:41:32.619 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:41:32.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:41:32.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:41:32.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:41:32.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:41:32.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:41:32.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:41:32.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:41:32.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:41:32.702 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:41:32.934 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:41:32.934 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T06:41:32.934 INFO:tasks.workunit.client.0.trial040.stderr:2 2026-02-13T06:41:32.934 INFO:tasks.workunit.client.0.trial040.stderr:3' 2026-02-13T06:41:32.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:41:32.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:32.935 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:41:32.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-13T06:41:32.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-13T06:41:32.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-13T06:41:32.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:32.999 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:41:33.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672967 2026-02-13T06:41:33.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672967 2026-02-13T06:41:33.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672967' 2026-02-13T06:41:33.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:33.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T06:41:33.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-13T06:41:33.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-13T06:41:33.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672967 2-60129542150' 2026-02-13T06:41:33.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:33.124 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-13T06:41:33.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-13T06:41:33.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-13T06:41:33.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672967 2-60129542150 3-77309411335' 2026-02-13T06:41:33.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:33.186 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-13T06:41:33.186 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:33.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:41:33.187 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-13T06:41:33.187 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:33.188 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-13T06:41:33.188 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-13T06:41:33.188 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836487 2026-02-13T06:41:33.189 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:41:33.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-13T06:41:33.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:41:34.415 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:41:34.415 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:41:34.639 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-13T06:41:34.639 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:34.640 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672967 2026-02-13T06:41:34.640 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:34.640 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:41:34.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672967 2026-02-13T06:41:34.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:34.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672967 2026-02-13T06:41:34.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-13T06:41:34.642 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672967 2026-02-13T06:41:34.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:41:34.866 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-13T06:41:34.866 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:34.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-13T06:41:34.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:34.867 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T06:41:34.868 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542150 2026-02-13T06:41:34.868 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:34.869 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542150 2026-02-13T06:41:34.869 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-13T06:41:34.870 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 60129542150 2026-02-13T06:41:34.870 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T06:41:35.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-13T06:41:35.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:35.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411335 2026-02-13T06:41:35.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:35.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-13T06:41:35.096 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-77309411335 2026-02-13T06:41:35.096 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:35.097 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411335 2026-02-13T06:41:35.097 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 77309411335' 2026-02-13T06:41:35.098 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.3 seq 77309411335 2026-02-13T06:41:35.098 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T06:41:35.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-13T06:41:35.321 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:41:35.322 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:41:35.322 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:41:35.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-13T06:41:35.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:41:35.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:41:35.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:41:35.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:41:35.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:41:35.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:41:35.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:41:35.858 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-13T06:41:35.859 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:41:35.859 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:41:35.859 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:41:36.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-13T06:41:36.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:41:36.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:41:36.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:261: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-13T06:41:36.395 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:36.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:262: TEST_0_osd: ceph osd ok-to-stop 1 2026-02-13T06:41:36.617 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:36.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:263: TEST_0_osd: ceph osd ok-to-stop 2 2026-02-13T06:41:36.842 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:36.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:264: TEST_0_osd: ceph osd ok-to-stop 3 2026-02-13T06:41:37.066 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:37.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:265: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-13T06:41:37.235 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":false,"osds":[0,1],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:37.235 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T06:41:37.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:266: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-13T06:41:37.394 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":false,"osds":[2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:37.395 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T06:41:37.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-13T06:41:37.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: grep '[0]' 2026-02-13T06:41:37.622 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:37.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: ceph osd ok-to-stop 1 --max 2 2026-02-13T06:41:37.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: grep '[1]' 2026-02-13T06:41:37.847 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:37.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:271: TEST_0_osd: ceph osd pool set ec min_size 2 2026-02-13T06:41:38.162 INFO:tasks.workunit.client.0.trial040.stderr:set pool 1 min_size to 2 2026-02-13T06:41:38.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:272: TEST_0_osd: wait_for_clean 2026-02-13T06:41:38.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:41:38.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:41:38.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:41:38.177 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:41:38.178 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:41:38.178 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:41:38.178 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:41:38.178 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:41:38.178 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:41:38.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:41:38.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:41:38.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:41:38.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:41:38.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:41:38.262 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:41:38.487 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:41:38.487 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T06:41:38.487 INFO:tasks.workunit.client.0.trial040.stderr:2 2026-02-13T06:41:38.487 INFO:tasks.workunit.client.0.trial040.stderr:3' 2026-02-13T06:41:38.487 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:41:38.487 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:38.487 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:41:38.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-13T06:41:38.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-13T06:41:38.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490' 2026-02-13T06:41:38.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:38.552 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:41:38.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-13T06:41:38.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-13T06:41:38.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490 1-42949672970' 2026-02-13T06:41:38.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:38.614 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T06:41:38.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-13T06:41:38.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-13T06:41:38.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490 1-42949672970 2-60129542153' 2026-02-13T06:41:38.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:38.676 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-13T06:41:38.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411338 2026-02-13T06:41:38.738 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411338 2026-02-13T06:41:38.738 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490 1-42949672970 2-60129542153 3-77309411338' 2026-02-13T06:41:38.738 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:38.738 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-13T06:41:38.738 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:38.738 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:41:38.739 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836490 2026-02-13T06:41:38.739 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:38.740 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836490 2026-02-13T06:41:38.740 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836490 2026-02-13T06:41:38.741 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-13T06:41:38.741 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:41:38.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T06:41:38.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:41:39.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:41:39.966 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:41:40.188 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T06:41:40.188 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:41:41.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T06:41:41.190 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:41:41.423 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-13T06:41:41.423 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:41.423 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-13T06:41:41.423 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:41.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:41:41.425 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672970 2026-02-13T06:41:41.425 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:41.426 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672970 2026-02-13T06:41:41.426 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-13T06:41:41.426 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672970 2026-02-13T06:41:41.426 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:41:41.654 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-13T06:41:41.654 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:41.655 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-13T06:41:41.655 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:41.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T06:41:41.656 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542153 2026-02-13T06:41:41.656 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:41.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542153 2026-02-13T06:41:41.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-13T06:41:41.658 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 60129542153 2026-02-13T06:41:41.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T06:41:41.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542154 -lt 60129542153 2026-02-13T06:41:41.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:41.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411338 2026-02-13T06:41:41.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:41.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-13T06:41:41.883 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-77309411338 2026-02-13T06:41:41.883 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:41.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411338 2026-02-13T06:41:41.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 77309411338' 2026-02-13T06:41:41.884 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.3 seq 77309411338 2026-02-13T06:41:41.884 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T06:41:42.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411338 -lt 77309411338 2026-02-13T06:41:42.106 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:41:42.107 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:41:42.107 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:41:42.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-13T06:41:42.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:41:42.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:41:42.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:41:42.421 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:41:42.421 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:41:42.421 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:41:42.421 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:41:42.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-13T06:41:42.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:41:42.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:41:42.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:41:42.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-13T06:41:42.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:41:42.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:41:42.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:274: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-13T06:41:43.185 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:43.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:275: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-13T06:41:43.408 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[2,3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:43.415 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:276: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-13T06:41:43.573 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:43.573 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T06:41:43.576 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:277: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-13T06:41:43.733 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:43.733 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T06:41:43.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-13T06:41:43.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: grep '[0,1]' 2026-02-13T06:41:43.960 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:43.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: ceph osd ok-to-stop 0 --max 20 2026-02-13T06:41:43.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: grep '[0,1]' 2026-02-13T06:41:44.183 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:44.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: ceph osd ok-to-stop 2 --max 2 2026-02-13T06:41:44.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: grep '[2,3]' 2026-02-13T06:41:44.412 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:44.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: ceph osd ok-to-stop 2 --max 20 2026-02-13T06:41:44.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: grep '[2,3]' 2026-02-13T06:41:44.636 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:44.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:285: TEST_0_osd: kill_daemons td/ok-to-stop TERM osd.0 2026-02-13T06:41:44.637 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:44.637 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:44.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:44.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:44.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:44.742 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:44.742 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:286: TEST_0_osd: ceph osd down 0 2026-02-13T06:41:44.969 INFO:tasks.workunit.client.0.trial040.stderr:osd.0 is already down. 2026-02-13T06:41:44.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:287: TEST_0_osd: wait_for_peered 2026-02-13T06:41:44.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1731: wait_for_peered: local cmd= 2026-02-13T06:41:44.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1732: wait_for_peered: local num_peered=-1 2026-02-13T06:41:44.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1733: wait_for_peered: local cur_peered 2026-02-13T06:41:44.978 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: get_timeout_delays 90 .1 2026-02-13T06:41:44.978 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:41:44.978 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:41:44.978 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:41:44.978 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:41:44.978 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:41:45.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:41:45.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: local -a delays 2026-02-13T06:41:45.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1735: wait_for_peered: local -i loop=0 2026-02-13T06:41:45.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1737: wait_for_peered: flush_pg_stats 2026-02-13T06:41:45.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:41:45.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:41:45.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:41:45.288 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T06:41:45.288 INFO:tasks.workunit.client.0.trial040.stderr:2 2026-02-13T06:41:45.288 INFO:tasks.workunit.client.0.trial040.stderr:3' 2026-02-13T06:41:45.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:41:45.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:45.288 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:41:45.335 INFO:tasks.workunit.client.0.trial040.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-13T06:41:45.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-13T06:41:45.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-13T06:41:45.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-13T06:41:45.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:45.335 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:41:45.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672973 2026-02-13T06:41:45.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672973 2026-02-13T06:41:45.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 1-42949672973' 2026-02-13T06:41:45.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:45.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T06:41:45.460 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542157 2026-02-13T06:41:45.460 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542157 2026-02-13T06:41:45.460 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 1-42949672973 2-60129542157' 2026-02-13T06:41:45.460 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:41:45.460 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-13T06:41:45.524 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411341 2026-02-13T06:41:45.524 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411341 2026-02-13T06:41:45.524 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 1-42949672973 2-60129542157 3-77309411341' 2026-02-13T06:41:45.525 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:45.525 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672973 2026-02-13T06:41:45.525 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:45.526 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:41:45.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672973 2026-02-13T06:41:45.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:45.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672973 2026-02-13T06:41:45.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-13T06:41:45.528 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672973 2026-02-13T06:41:45.528 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:41:45.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672971 -lt 42949672973 2026-02-13T06:41:45.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:41:46.761 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:41:46.761 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:41:46.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672973 -lt 42949672973 2026-02-13T06:41:46.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:46.989 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542157 2026-02-13T06:41:46.990 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:46.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T06:41:46.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542157 2026-02-13T06:41:46.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:46.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542157 2026-02-13T06:41:46.992 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 60129542157 2026-02-13T06:41:46.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-13T06:41:46.993 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T06:41:47.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542157 -lt 60129542157 2026-02-13T06:41:47.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:41:47.217 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411341 2026-02-13T06:41:47.217 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:41:47.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-13T06:41:47.218 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-77309411341 2026-02-13T06:41:47.218 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:41:47.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411341 2026-02-13T06:41:47.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 77309411341' 2026-02-13T06:41:47.220 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.3 seq 77309411341 2026-02-13T06:41:47.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T06:41:47.443 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411341 -lt 77309411341 2026-02-13T06:41:47.444 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: get_num_pgs 2026-02-13T06:41:47.444 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:41:47.444 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:41:47.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: test 32 == 0 2026-02-13T06:41:47.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1742: wait_for_peered: true 2026-02-13T06:41:47.756 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: get_num_active_or_peered 2026-02-13T06:41:47.756 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-13T06:41:47.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-13T06:41:47.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-13T06:41:47.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-13T06:41:47.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-13T06:41:47.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: cur_peered=32 2026-02-13T06:41:47.979 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: get_num_pgs 2026-02-13T06:41:47.979 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:41:47.979 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:41:48.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: test 32 = 32 2026-02-13T06:41:48.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: break 2026-02-13T06:41:48.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1762: wait_for_peered: return 0 2026-02-13T06:41:48.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:289: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-13T06:41:48.504 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":0,"num_not_ok_pgs":0} 2026-02-13T06:41:48.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:290: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-13T06:41:48.728 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:48.736 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:291: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-13T06:41:48.892 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:48.892 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T06:41:48.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:292: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-13T06:41:49.051 INFO:tasks.workunit.client.0.trial040.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T06:41:49.051 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T06:41:49.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:49.055 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:49.055 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:49.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:49.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:49.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:49.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:49.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T06:41:49.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:41:49.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:49.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:49.165 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:49.165 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:49.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:49.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:49.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:49.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:49.171 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:49.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:49.172 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:49.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:49.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:49.173 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:49.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:49.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:49.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:49.175 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:49.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:49.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:49.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:41:49.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:49.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.192 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:41:49.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:49.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:49.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:49.196 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:49.196 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:49.196 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:49.197 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:49.197 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:49.197 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:49.197 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:49.198 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:49.198 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:49.198 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:49.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:49.199 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:49.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:49.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:49.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:41:49.201 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:49.202 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.202 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:41:49.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:49.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:49.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T06:41:49.203 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:41:49.204 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.204 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.43705 2026-02-13T06:41:49.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:41:49.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:41:49.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:41:49.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T06:41:49.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_1_mon_checks td/ok-to-stop 2026-02-13T06:41:49.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:28: TEST_1_mon_checks: local dir=td/ok-to-stop 2026-02-13T06:41:49.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:30: TEST_1_mon_checks: CEPH_ARGS='--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T06:41:49.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:32: TEST_1_mon_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T06:41:49.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:41:49.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:41:49.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:41:49.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:41:49.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T06:41:49.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T06:41:49.226 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:41:49.226 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:49.226 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:49.226 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:49.226 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.226 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:49.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T06:41:49.252 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:41:49.253 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:41:49.253 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:49.253 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:49.254 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:41:49.255 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:41:49.255 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:49.255 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:49.255 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:49.255 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:49.255 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.255 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.255 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:49.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:49.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get fsid 2026-02-13T06:41:49.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:41:49.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:49.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:49.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:41:49.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:41:49.301 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:49.301 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:49.301 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:49.301 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:49.301 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.301 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.301 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:49.301 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:49.301 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get mon_host 2026-02-13T06:41:49.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:34: TEST_1_mon_checks: ceph mon ok-to-stop dne 2026-02-13T06:41:49.439 INFO:tasks.workunit.client.0.trial040.stderr:invalid mon(s) specified: [dne] 2026-02-13T06:41:49.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:35: TEST_1_mon_checks: ceph mon ok-to-stop a 2026-02-13T06:41:49.520 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a] 2026-02-13T06:41:49.523 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:37: TEST_1_mon_checks: ceph mon ok-to-add-offline 2026-02-13T06:41:49.598 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-13T06:41:49.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:39: TEST_1_mon_checks: ceph mon ok-to-rm a 2026-02-13T06:41:49.676 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-13T06:41:49.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:40: TEST_1_mon_checks: ceph mon ok-to-rm dne 2026-02-13T06:41:49.773 INFO:tasks.workunit.client.0.trial040.stderr:mon.dne does not exist 2026-02-13T06:41:49.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:49.779 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:49.780 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:49.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:49.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:49.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:49.886 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:49.886 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T06:41:49.887 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:41:49.887 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:49.887 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:49.887 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:49.887 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:49.887 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:49.887 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:49.887 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:49.891 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:49.892 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:49.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:49.892 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:49.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:49.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:49.894 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:49.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:49.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:49.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:49.895 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:49.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:49.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:49.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:41:49.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:49.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:41:49.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:49.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:49.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T06:41:49.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T06:41:49.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T06:41:49.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T06:41:49.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:41:49.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:49.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:49.901 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:49.901 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:49.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:49.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:49.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:49.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:49.902 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:49.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:49.903 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:49.903 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:49.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:49.904 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:49.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:49.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:49.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:49.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:49.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:49.907 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:49.907 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:41:49.908 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:49.908 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.908 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.908 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:41:49.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:49.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:49.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T06:41:49.910 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:41:49.910 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.43705 2026-02-13T06:41:49.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:41:49.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:41:49.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:41:49.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T06:41:49.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_2_mons_checks td/ok-to-stop 2026-02-13T06:41:49.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:44: TEST_2_mons_checks: local dir=td/ok-to-stop 2026-02-13T06:41:49.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:46: TEST_2_mons_checks: CEPH_ARGS='--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151 ' 2026-02-13T06:41:49.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:48: TEST_2_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T06:41:49.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:41:49.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:41:49.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:41:49.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:41:49.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T06:41:49.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T06:41:49.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:41:49.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:49.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:49.932 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:49.933 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.933 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:49.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T06:41:49.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:41:49.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:41:49.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:49.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:49.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:41:49.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:41:49.960 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:49.960 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:49.960 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:49.960 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:49.960 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:49.960 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:49.960 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:49.960 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:49.961 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get fsid 2026-02-13T06:41:50.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:41:50.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:50.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:50.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:41:50.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:41:50.005 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:50.005 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:50.006 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:50.006 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:50.007 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:50.007 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:50.007 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:50.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:50.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get mon_host 2026-02-13T06:41:50.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:49: TEST_2_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-13T06:41:50.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:41:50.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:41:50.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T06:41:50.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:41:50.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-13T06:41:50.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-13T06:41:50.068 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:41:50.068 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:50.068 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:50.069 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:50.069 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:50.069 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:50.069 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:50.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-13T06:41:50.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:41:50.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T06:41:50.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:50.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T06:41:50.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:41:50.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:41:50.097 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T06:41:50.097 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T06:41:50.097 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T06:41:50.097 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:50.097 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:50.097 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:50.097 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.b.asok 2026-02-13T06:41:50.097 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:50.097 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.b.asok config get fsid 2026-02-13T06:41:50.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T06:41:50.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:50.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T06:41:50.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:41:50.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:41:50.142 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T06:41:50.142 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T06:41:50.142 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T06:41:50.142 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:50.142 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:50.142 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:50.142 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.b.asok 2026-02-13T06:41:50.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:50.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.b.asok config get mon_host 2026-02-13T06:41:50.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:51: TEST_2_mons_checks: ceph mon ok-to-stop dne 2026-02-13T06:41:50.281 INFO:tasks.workunit.client.0.trial040.stderr:invalid mon(s) specified: [dne] 2026-02-13T06:41:50.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:52: TEST_2_mons_checks: ceph mon ok-to-stop a 2026-02-13T06:41:50.362 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (b) after stopping mons [a] 2026-02-13T06:41:50.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:53: TEST_2_mons_checks: ceph mon ok-to-stop b 2026-02-13T06:41:50.440 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b] 2026-02-13T06:41:50.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:54: TEST_2_mons_checks: ceph mon ok-to-stop a b 2026-02-13T06:41:50.520 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b] 2026-02-13T06:41:50.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:56: TEST_2_mons_checks: ceph mon ok-to-add-offline 2026-02-13T06:41:50.615 INFO:tasks.workunit.client.0.trial040.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T06:41:50.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:58: TEST_2_mons_checks: ceph mon ok-to-rm a 2026-02-13T06:41:50.718 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.a 2026-02-13T06:41:50.725 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:59: TEST_2_mons_checks: ceph mon ok-to-rm b 2026-02-13T06:41:50.819 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.b 2026-02-13T06:41:50.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:60: TEST_2_mons_checks: ceph mon ok-to-rm dne 2026-02-13T06:41:50.921 INFO:tasks.workunit.client.0.trial040.stderr:mon.dne does not exist 2026-02-13T06:41:50.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:50.928 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:50.928 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:50.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:50.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:50.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:51.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:51.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T06:41:51.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:41:51.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:51.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:51.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:51.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:51.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:51.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:51.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:51.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:51.041 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:51.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:51.041 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:51.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:51.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:51.043 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:51.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:51.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:51.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:51.044 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:51.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:51.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:51.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:41:51.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:51.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:41:51.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:51.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:51.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T06:41:51.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T06:41:51.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T06:41:51.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T06:41:51.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:41:51.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:41:51.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:41:51.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:51.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:51.053 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:51.053 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:51.053 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:51.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:51.056 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:41:51.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:41:51.056 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:41:51.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:41:51.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:41:51.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:41:51.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:51.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:41:51.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:41:51.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:41:51.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:41:51.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:41:51.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:41:51.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:41:51.060 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.060 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:41:51.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:41:51.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:41:51.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T06:41:51.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:41:51.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.43705 2026-02-13T06:41:51.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:41:51.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:41:51.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:41:51.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T06:41:51.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_3_mons_checks td/ok-to-stop 2026-02-13T06:41:51.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:64: TEST_3_mons_checks: local dir=td/ok-to-stop 2026-02-13T06:41:51.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:66: TEST_3_mons_checks: CEPH_ARGS='--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152 ' 2026-02-13T06:41:51.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:68: TEST_3_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T06:41:51.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:41:51.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:41:51.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:41:51.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:41:51.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T06:41:51.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T06:41:51.084 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:41:51.084 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:51.084 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:51.084 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:51.084 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.084 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.084 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:51.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T06:41:51.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:41:51.111 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:41:51.111 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:51.111 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:51.111 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:51.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get fsid 2026-02-13T06:41:51.160 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:41:51.160 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:51.160 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:41:51.160 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:41:51.160 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:41:51.160 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:41:51.161 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:41:51.161 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:41:51.161 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:51.161 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.161 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.161 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:41:51.161 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:51.161 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get mon_host 2026-02-13T06:41:51.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:69: TEST_3_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-13T06:41:51.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:41:51.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:41:51.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T06:41:51.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:41:51.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-13T06:41:51.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-13T06:41:51.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:41:51.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:51.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:51.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:51.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:51.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-13T06:41:51.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:41:51.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T06:41:51.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:51.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T06:41:51.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:41:51.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:41:51.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T06:41:51.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T06:41:51.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T06:41:51.250 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:51.250 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.250 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.b.asok 2026-02-13T06:41:51.251 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:51.251 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.b.asok config get fsid 2026-02-13T06:41:51.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T06:41:51.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:51.296 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T06:41:51.296 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:41:51.296 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:41:51.296 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T06:41:51.296 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T06:41:51.296 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T06:41:51.296 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:51.296 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.296 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.297 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.b.asok 2026-02-13T06:41:51.297 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:51.297 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.b.asok config get mon_host 2026-02-13T06:41:51.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:70: TEST_3_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-13T06:41:51.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:41:51.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:41:51.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-13T06:41:51.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:41:51.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-13T06:41:51.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-13T06:41:51.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:41:51.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:41:51.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:41:51.360 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:41:51.360 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.360 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:41:51.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-13T06:41:51.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:41:51.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-13T06:41:51.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:51.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T06:41:51.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:41:51.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:41:51.389 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T06:41:51.389 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T06:41:51.389 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T06:41:51.389 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:51.389 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.389 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.389 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.c.asok 2026-02-13T06:41:51.389 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:51.389 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.c.asok config get fsid 2026-02-13T06:41:51.433 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-13T06:41:51.433 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:41:51.433 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T06:41:51.433 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:41:51.434 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:41:51.434 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T06:41:51.434 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T06:41:51.434 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T06:41:51.434 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:41:51.435 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:41:51.435 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:41:51.435 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.c.asok 2026-02-13T06:41:51.435 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:41:51.435 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.c.asok config get mon_host 2026-02-13T06:41:51.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:71: TEST_3_mons_checks: wait_for_quorum 60 3 2026-02-13T06:41:51.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T06:41:51.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-13T06:41:51.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T06:41:51.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-13T06:41:51.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T06:41:51.479 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T06:41:51.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770964971 2026-02-13T06:41:51.481 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:41:51.482 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964911 -lt 1770964971 ]] 2026-02-13T06:41:51.482 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:41:51.482 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:41:57.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:41:57.586 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:41:57.586 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:41:57.586 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:41:57.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T06:41:57.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T06:41:57.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T06:41:57.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T06:41:57.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T06:41:57.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:73: TEST_3_mons_checks: ceph mon ok-to-stop dne 2026-02-13T06:41:57.684 INFO:tasks.workunit.client.0.trial040.stderr:invalid mon(s) specified: [dne] 2026-02-13T06:41:57.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:74: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-13T06:41:57.785 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,c) after stopping [a] 2026-02-13T06:41:57.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:75: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-13T06:41:57.886 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-13T06:41:57.891 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:76: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-13T06:41:57.986 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,b) after stopping [c] 2026-02-13T06:41:57.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:77: TEST_3_mons_checks: ceph mon ok-to-stop a b 2026-02-13T06:41:58.067 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a,b] 2026-02-13T06:41:58.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:78: TEST_3_mons_checks: ceph mon ok-to-stop b c 2026-02-13T06:41:58.145 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b,c] 2026-02-13T06:41:58.147 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:79: TEST_3_mons_checks: ceph mon ok-to-stop a b c 2026-02-13T06:41:58.223 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b,c] 2026-02-13T06:41:58.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:81: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-13T06:41:58.319 INFO:tasks.workunit.client.0.trial040.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T06:41:58.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:83: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-13T06:41:58.418 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.a 2026-02-13T06:41:58.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:84: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-13T06:41:58.519 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.b 2026-02-13T06:41:58.525 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:85: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-13T06:41:58.617 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.c 2026-02-13T06:41:58.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:87: TEST_3_mons_checks: kill_daemons td/ok-to-stop KILL mon.b 2026-02-13T06:41:58.623 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:41:58.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:41:58.624 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:41:58.624 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:41:58.624 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:41:58.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:41:58.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:88: TEST_3_mons_checks: wait_for_quorum 60 2 2026-02-13T06:41:58.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T06:41:58.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=2 2026-02-13T06:41:58.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T06:41:58.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 2 ]] 2026-02-13T06:41:58.731 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T06:41:58.731 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T06:41:58.731 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770964978 2026-02-13T06:41:58.732 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:41:58.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964918 -lt 1770964978 ]] 2026-02-13T06:41:58.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:41:58.733 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:41:58.836 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:41:58.836 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:41:58.836 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:41:58.836 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:41:58.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:41:58.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:41:58.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:41:58.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964918 -lt 1770964978 ]] 2026-02-13T06:41:58.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:41:58.840 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:41:58.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:41:58.940 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:41:58.941 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:41:58.941 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:41:58.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:41:58.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:41:58.943 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:41:58.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964918 -lt 1770964978 ]] 2026-02-13T06:41:58.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:41:58.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:41:59.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:41:59.048 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:41:59.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:41:59.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:41:59.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:41:59.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:41:59.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:41:59.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964919 -lt 1770964978 ]] 2026-02-13T06:41:59.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:41:59.052 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:02.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:02.154 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.155 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.155 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:02.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:02.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:02.157 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:02.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964922 -lt 1770964978 ]] 2026-02-13T06:42:02.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:02.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:02.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:02.259 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:02.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:02.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:02.262 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:02.263 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964922 -lt 1770964978 ]] 2026-02-13T06:42:02.263 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:02.263 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:02.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:02.367 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.368 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.368 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:02.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:02.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:02.371 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:02.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964922 -lt 1770964978 ]] 2026-02-13T06:42:02.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:02.372 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:02.471 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:02.471 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.472 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.472 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:02.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:02.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:02.474 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:02.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964922 -lt 1770964978 ]] 2026-02-13T06:42:02.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:02.475 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:02.577 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:02.577 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.578 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:02.578 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:02.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:02.580 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:02.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964922 -lt 1770964978 ]] 2026-02-13T06:42:02.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:02.581 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:02.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:02.684 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.684 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.684 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:02.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:02.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:02.687 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:02.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964922 -lt 1770964978 ]] 2026-02-13T06:42:02.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:02.688 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:02.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:02.789 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:02.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:02.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:02.792 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:02.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964922 -lt 1770964978 ]] 2026-02-13T06:42:02.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:02.793 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:02.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:02.897 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:02.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:02.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:02.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:02.900 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:02.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964922 -lt 1770964978 ]] 2026-02-13T06:42:02.901 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:02.901 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:06.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:06.003 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:06.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:06.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:06.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:06.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:06.006 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:06.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964926 -lt 1770964978 ]] 2026-02-13T06:42:06.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:06.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:06.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:06.112 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:06.113 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:06.113 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:06.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:06.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:06.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:06.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964926 -lt 1770964978 ]] 2026-02-13T06:42:06.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T06:42:06.116 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:18.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:18.220 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":12,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:18.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:41:51.075580Z","created":"2026-02-13T06:41:51.075580Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:18.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T06:42:18.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T06:42:18.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T06:42:18.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T06:42:18.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T06:42:18.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T06:42:18.223 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:90: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-13T06:42:18.298 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a] 2026-02-13T06:42:18.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:91: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-13T06:42:21.395 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-13T06:42:21.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:92: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-13T06:42:21.474 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [c] 2026-02-13T06:42:21.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:94: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-13T06:42:21.549 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-13T06:42:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:96: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-13T06:42:21.627 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-13T06:42:21.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:97: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-13T06:42:21.725 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.b 2026-02-13T06:42:21.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:98: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-13T06:42:21.807 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-13T06:42:21.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T06:42:21.810 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:42:21.810 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:42:21.811 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:42:21.811 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:42:21.811 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:42:21.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:42:21.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T06:42:21.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:42:21.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:42:21.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:42:21.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:42:21.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:42:21.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:42:21.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:42:21.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:42:21.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:42:21.923 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:42:21.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:42:21.924 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:42:21.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:42:21.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:42:21.926 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:42:21.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:42:21.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:42:21.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:42:21.927 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:42:21.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:42:21.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:42:21.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:42:21.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:42:21.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:21.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:21.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:42:21.934 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:42:21.934 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:42:21.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:42:21.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:42:21.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:42:21.936 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:42:21.937 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:42:21.937 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:42:21.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:42:21.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:42:21.938 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:42:21.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:42:21.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:42:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:42:21.940 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:42:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:42:21.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:42:21.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:42:21.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:42:21.943 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:21.943 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:21.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:42:21.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:42:21.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:42:21.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T06:42:21.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:42:21.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:21.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:21.945 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.43705 2026-02-13T06:42:21.946 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:42:21.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:42:21.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:42:21.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T06:42:21.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_4_mons_checks td/ok-to-stop 2026-02-13T06:42:21.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:102: TEST_4_mons_checks: local dir=td/ok-to-stop 2026-02-13T06:42:21.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:104: TEST_4_mons_checks: CEPH_ARGS='--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153 ' 2026-02-13T06:42:21.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:106: TEST_4_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T06:42:21.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:42:21.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:42:21.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:42:21.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:42:21.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T06:42:21.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T06:42:21.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:42:21.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:42:21.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:42:21.967 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:42:21.967 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:21.968 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:21.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:42:21.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T06:42:21.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:42:21.994 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:42:21.994 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:21.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:42:21.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:42:21.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:42:21.996 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:42:21.996 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:42:21.996 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:42:21.996 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:21.996 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:21.996 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:21.997 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:42:21.997 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:21.997 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get fsid 2026-02-13T06:42:22.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:42:22.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:22.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:42:22.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:42:22.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:42:22.041 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:42:22.042 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:42:22.042 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:42:22.042 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:22.042 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.042 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.042 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:42:22.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:22.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get mon_host 2026-02-13T06:42:22.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:107: TEST_4_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-13T06:42:22.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:42:22.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:42:22.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T06:42:22.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:42:22.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-13T06:42:22.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-13T06:42:22.104 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:42:22.104 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:42:22.104 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:42:22.104 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:42:22.104 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.104 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.105 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:42:22.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-13T06:42:22.131 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:42:22.132 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T06:42:22.132 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:22.132 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T06:42:22.132 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:42:22.132 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:42:22.133 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T06:42:22.134 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T06:42:22.134 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T06:42:22.134 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:22.134 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.134 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.134 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.b.asok 2026-02-13T06:42:22.134 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:22.134 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.b.asok config get fsid 2026-02-13T06:42:22.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T06:42:22.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:22.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T06:42:22.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:42:22.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:42:22.183 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T06:42:22.183 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T06:42:22.183 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T06:42:22.183 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:22.184 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.184 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.184 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.b.asok 2026-02-13T06:42:22.184 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:22.184 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.b.asok config get mon_host 2026-02-13T06:42:22.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:108: TEST_4_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-13T06:42:22.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:42:22.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:42:22.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-13T06:42:22.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:42:22.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-13T06:42:22.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-13T06:42:22.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:42:22.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:42:22.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:42:22.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:42:22.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:42:22.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-13T06:42:22.276 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:42:22.277 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-13T06:42:22.277 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:22.277 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T06:42:22.277 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:42:22.277 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:42:22.278 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T06:42:22.279 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T06:42:22.279 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T06:42:22.279 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:22.279 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.279 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.279 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.c.asok 2026-02-13T06:42:22.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:22.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.c.asok config get fsid 2026-02-13T06:42:22.323 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-13T06:42:22.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:22.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T06:42:22.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:42:22.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:42:22.325 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T06:42:22.325 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T06:42:22.325 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T06:42:22.325 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:22.325 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.326 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.326 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.c.asok 2026-02-13T06:42:22.326 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:22.326 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.c.asok config get mon_host 2026-02-13T06:42:22.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:109: TEST_4_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-13T06:42:22.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:42:22.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:42:22.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-13T06:42:22.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:42:22.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-13T06:42:22.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-13T06:42:22.389 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:42:22.389 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:42:22.389 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:42:22.389 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:42:22.389 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.390 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.390 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:42:22.390 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-13T06:42:22.415 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:42:22.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-13T06:42:22.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:22.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-13T06:42:22.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:42:22.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:42:22.418 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-13T06:42:22.418 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-13T06:42:22.418 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-13T06:42:22.418 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:22.419 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.419 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.419 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.d.asok 2026-02-13T06:42:22.419 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:22.419 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.d.asok config get fsid 2026-02-13T06:42:22.463 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:22.464 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.d.asok 2026-02-13T06:42:22.465 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:22.465 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.d.asok config get mon_host 2026-02-13T06:42:22.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:110: TEST_4_mons_checks: wait_for_quorum 60 4 2026-02-13T06:42:22.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T06:42:22.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-13T06:42:22.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T06:42:22.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-13T06:42:22.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T06:42:22.509 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T06:42:22.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770965002 2026-02-13T06:42:22.510 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:22.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964942 -lt 1770965002 ]] 2026-02-13T06:42:22.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:42:22.511 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:22.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:22.813 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:22.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:42:22.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:22.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T06:42:22.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T06:42:22.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T06:42:22.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T06:42:22.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T06:42:22.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:112: TEST_4_mons_checks: ceph mon ok-to-stop dne 2026-02-13T06:42:22.911 INFO:tasks.workunit.client.0.trial040.stderr:invalid mon(s) specified: [dne] 2026-02-13T06:42:22.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:113: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-13T06:42:23.010 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-13T06:42:23.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:114: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-13T06:42:23.110 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,c,d) after stopping [b] 2026-02-13T06:42:23.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:115: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-13T06:42:23.214 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,b,d) after stopping [c] 2026-02-13T06:42:23.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:116: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-13T06:42:23.319 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,b,c) after stopping [d] 2026-02-13T06:42:23.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:117: TEST_4_mons_checks: ceph mon ok-to-stop a b 2026-02-13T06:42:23.402 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [a,b] 2026-02-13T06:42:23.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:118: TEST_4_mons_checks: ceph mon ok-to-stop c d 2026-02-13T06:42:23.481 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (a,b) after stopping mons [c,d] 2026-02-13T06:42:23.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:120: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-13T06:42:23.576 INFO:tasks.workunit.client.0.trial040.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T06:42:23.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:122: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-13T06:42:23.676 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.a 2026-02-13T06:42:23.682 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:123: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-13T06:42:23.781 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.b 2026-02-13T06:42:23.787 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:124: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-13T06:42:23.885 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.c 2026-02-13T06:42:23.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:126: TEST_4_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-13T06:42:23.893 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:42:23.893 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:42:23.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:42:23.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:42:23.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:42:23.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:42:23.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:127: TEST_4_mons_checks: wait_for_quorum 60 3 2026-02-13T06:42:24.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T06:42:24.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-13T06:42:24.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T06:42:24.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-13T06:42:24.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T06:42:24.000 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T06:42:24.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770965003 2026-02-13T06:42:24.001 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:24.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964944 -lt 1770965003 ]] 2026-02-13T06:42:24.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:42:24.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:24.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:24.103 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:24.104 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:24.104 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:42:24.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:24.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:24.106 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:24.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964944 -lt 1770965003 ]] 2026-02-13T06:42:24.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:42:24.107 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:27.212 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:27.212 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:27.213 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:27.213 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:42:27.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:27.215 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:27.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:27.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964947 -lt 1770965003 ]] 2026-02-13T06:42:27.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:42:27.216 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:27.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:27.318 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:27.318 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:27.319 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:42:27.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:27.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:27.321 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:27.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964947 -lt 1770965003 ]] 2026-02-13T06:42:27.322 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:42:27.322 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:27.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:27.425 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:27.425 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:27.425 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:42:27.427 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:27.427 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:27.427 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:27.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964947 -lt 1770965003 ]] 2026-02-13T06:42:27.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:42:27.429 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:30.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:30.531 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:30.532 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:30.532 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:42:30.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:30.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:30.534 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:30.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964950 -lt 1770965003 ]] 2026-02-13T06:42:30.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:42:30.535 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:30.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:30.637 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:30.637 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:30.638 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:42:30.639 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:30.640 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:30.640 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:30.641 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964950 -lt 1770965003 ]] 2026-02-13T06:42:30.641 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:42:30.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:30.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:30.743 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:30.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:30.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:42:30.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:42:30.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:42:30.746 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:30.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964950 -lt 1770965003 ]] 2026-02-13T06:42:30.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:42:30.747 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:42:38.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:42:38.852 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":6,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:38.853 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:21.959276Z","created":"2026-02-13T06:42:21.959276Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:42:38.853 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:42:38.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T06:42:38.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T06:42:38.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T06:42:38.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T06:42:38.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T06:42:38.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:129: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-13T06:42:38.950 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-13T06:42:38.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:130: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-13T06:42:42.035 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-13T06:42:42.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:131: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-13T06:42:42.113 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-13T06:42:42.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:132: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-13T06:42:42.193 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-13T06:42:42.196 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:134: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-13T06:42:45.293 INFO:tasks.workunit.client.0.trial040.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T06:42:45.299 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:136: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-13T06:42:45.393 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.a 2026-02-13T06:42:45.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:137: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-13T06:42:48.497 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.b 2026-02-13T06:42:48.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:138: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-13T06:42:51.602 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.c 2026-02-13T06:42:51.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:139: TEST_4_mons_checks: ceph mon ok-to-rm d 2026-02-13T06:42:51.704 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.d 2026-02-13T06:42:51.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T06:42:51.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:42:51.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:42:51.712 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:42:51.712 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:42:51.712 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:42:51.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:42:51.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T06:42:51.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:42:51.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:42:51.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:42:51.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:42:51.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:42:51.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:42:51.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:42:51.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:42:51.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:42:51.825 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:42:51.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:42:51.826 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:42:51.827 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:42:51.827 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:42:51.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:42:51.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:42:51.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:42:51.829 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:42:51.829 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:42:51.829 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:42:51.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:42:51.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:42:51.835 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:42:51.835 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:51.835 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:51.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:42:51.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:42:51.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:42:51.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T06:42:51.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:42:51.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:42:51.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:42:51.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:42:51.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:42:51.840 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:42:51.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:42:51.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:42:51.841 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:42:51.842 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:42:51.842 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:42:51.842 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:42:51.842 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:42:51.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:42:51.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:42:51.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:42:51.845 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:42:51.845 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:51.845 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:51.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:42:51.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:42:51.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:42:51.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T06:42:51.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:42:51.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:51.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:51.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.43705 2026-02-13T06:42:51.848 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:42:51.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:42:51.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:42:51.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T06:42:51.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_5_mons_checks td/ok-to-stop 2026-02-13T06:42:51.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:143: TEST_5_mons_checks: local dir=td/ok-to-stop 2026-02-13T06:42:51.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:145: TEST_5_mons_checks: CEPH_ARGS='--fsid=ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153,127.0.0.1:7154 ' 2026-02-13T06:42:51.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:147: TEST_5_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T06:42:51.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:42:51.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:42:51.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:42:51.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:42:51.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T06:42:51.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T06:42:51.869 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:42:51.869 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:42:51.870 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:42:51.870 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:42:51.870 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:51.870 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:51.870 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:42:51.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T06:42:51.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:42:51.897 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:42:51.897 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:51.897 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:42:51.897 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:42:51.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:42:51.898 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:42:51.899 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:42:51.899 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:42:51.899 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:51.899 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:51.899 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:51.899 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:42:51.899 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:51.899 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get fsid 2026-02-13T06:42:51.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:42:51.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:51.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:42:51.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:42:51.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:42:51.943 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:42:51.943 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:42:51.943 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:42:51.944 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:51.944 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:51.944 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:51.944 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.a.asok 2026-02-13T06:42:51.944 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:51.944 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.a.asok config get mon_host 2026-02-13T06:42:51.987 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:148: TEST_5_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-13T06:42:51.987 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:42:51.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:42:51.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T06:42:51.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:42:51.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-13T06:42:51.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-13T06:42:52.006 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:42:52.006 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:42:52.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:42:52.007 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:42:52.007 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.007 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:42:52.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-13T06:42:52.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:42:52.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T06:42:52.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:52.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T06:42:52.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:42:52.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:42:52.034 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T06:42:52.034 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T06:42:52.034 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T06:42:52.034 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:52.034 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.035 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.035 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.b.asok 2026-02-13T06:42:52.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:52.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.b.asok config get fsid 2026-02-13T06:42:52.078 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T06:42:52.078 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:52.079 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T06:42:52.079 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:42:52.079 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:42:52.080 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T06:42:52.080 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T06:42:52.080 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T06:42:52.080 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:52.080 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.080 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.080 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.b.asok 2026-02-13T06:42:52.081 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:52.081 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.b.asok config get mon_host 2026-02-13T06:42:52.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:149: TEST_5_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-13T06:42:52.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:42:52.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:42:52.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-13T06:42:52.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:42:52.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-13T06:42:52.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-13T06:42:52.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:42:52.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:42:52.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:42:52.146 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:42:52.146 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.147 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.147 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:42:52.147 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-13T06:42:52.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:42:52.173 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-13T06:42:52.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:52.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T06:42:52.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:42:52.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:42:52.175 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T06:42:52.175 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T06:42:52.175 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T06:42:52.175 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:52.175 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.175 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.176 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.c.asok 2026-02-13T06:42:52.176 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:52.176 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.c.asok config get fsid 2026-02-13T06:42:52.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-13T06:42:52.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:52.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T06:42:52.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:42:52.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:42:52.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T06:42:52.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T06:42:52.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T06:42:52.222 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:52.222 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.222 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.c.asok 2026-02-13T06:42:52.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:52.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.c.asok config get mon_host 2026-02-13T06:42:52.265 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:150: TEST_5_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-13T06:42:52.265 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:42:52.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:42:52.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-13T06:42:52.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:42:52.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-13T06:42:52.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-13T06:42:52.285 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:42:52.285 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:42:52.286 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:42:52.286 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:42:52.286 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.286 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.286 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:42:52.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-13T06:42:52.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:42:52.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-13T06:42:52.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:52.313 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-13T06:42:52.313 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:42:52.313 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:42:52.314 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-13T06:42:52.314 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-13T06:42:52.314 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-13T06:42:52.314 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:52.314 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.314 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.314 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.d.asok 2026-02-13T06:42:52.314 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:52.315 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.d.asok config get fsid 2026-02-13T06:42:52.358 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-13T06:42:52.358 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:52.358 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-13T06:42:52.358 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:42:52.358 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:42:52.359 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-13T06:42:52.360 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-13T06:42:52.360 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-13T06:42:52.360 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:52.360 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.360 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.360 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.d.asok 2026-02-13T06:42:52.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:52.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.d.asok config get mon_host 2026-02-13T06:42:52.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:151: TEST_5_mons_checks: run_mon td/ok-to-stop e --public-addr=127.0.0.1:7154 2026-02-13T06:42:52.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T06:42:52.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:42:52.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=e 2026-02-13T06:42:52.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:42:52.404 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/e 2026-02-13T06:42:52.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id e --mkfs --mon-data=td/ok-to-stop/e --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7154 2026-02-13T06:42:52.423 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:42:52.423 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:42:52.423 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:42:52.423 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:42:52.423 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.423 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.423 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.43705/$cluster-$name.asok' 2026-02-13T06:42:52.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id e --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/e '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.43705/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7154 2026-02-13T06:42:52.449 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:42:52.450 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e fsid 2026-02-13T06:42:52.450 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:52.450 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-13T06:42:52.451 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:42:52.451 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:42:52.452 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-13T06:42:52.452 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-13T06:42:52.452 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-13T06:42:52.452 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:52.452 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.452 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.452 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.e.asok 2026-02-13T06:42:52.452 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:52.453 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.e.asok config get fsid 2026-02-13T06:42:52.499 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e mon_host 2026-02-13T06:42:52.499 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:42:52.499 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-13T06:42:52.499 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.43705/ceph-mon.e.asok 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:42:52.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.43705/ceph-mon.e.asok config get mon_host 2026-02-13T06:42:52.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:152: TEST_5_mons_checks: wait_for_quorum 60 5 2026-02-13T06:42:52.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T06:42:52.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=5 2026-02-13T06:42:52.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T06:42:52.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 5 ]] 2026-02-13T06:42:52.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T06:42:52.547 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T06:42:52.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770965032 2026-02-13T06:42:52.548 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:42:52.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964972 -lt 1770965032 ]] 2026-02-13T06:42:52.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-13T06:42:52.549 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:04.652 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:04.652 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:04.653 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:04.653 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-13T06:43:04.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T06:43:04.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T06:43:04.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T06:43:04.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T06:43:04.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T06:43:04.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:154: TEST_5_mons_checks: ceph mon ok-to-stop dne 2026-02-13T06:43:04.748 INFO:tasks.workunit.client.0.trial040.stderr:invalid mon(s) specified: [dne] 2026-02-13T06:43:04.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:155: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-13T06:43:04.847 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-13T06:43:04.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:156: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-13T06:43:04.946 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,c,d,e) after stopping [b] 2026-02-13T06:43:04.953 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:157: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-13T06:43:05.048 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,b,d,e) after stopping [c] 2026-02-13T06:43:05.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:158: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-13T06:43:05.149 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,b,c,e) after stopping [d] 2026-02-13T06:43:05.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:159: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-13T06:43:05.249 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,b,c,d) after stopping [e] 2026-02-13T06:43:05.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:160: TEST_5_mons_checks: ceph mon ok-to-stop a b 2026-02-13T06:43:05.349 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (c,d,e) after stopping [a,b] 2026-02-13T06:43:05.355 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:161: TEST_5_mons_checks: ceph mon ok-to-stop c d 2026-02-13T06:43:05.451 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (a,b,e) after stopping [c,d] 2026-02-13T06:43:05.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:162: TEST_5_mons_checks: ceph mon ok-to-stop a b c 2026-02-13T06:43:05.535 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (d,e) after stopping mons [a,b,c] 2026-02-13T06:43:05.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:164: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-13T06:43:05.631 INFO:tasks.workunit.client.0.trial040.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T06:43:05.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:166: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-13T06:43:05.731 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.a 2026-02-13T06:43:05.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:167: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-13T06:43:05.833 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.b 2026-02-13T06:43:05.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:168: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-13T06:43:08.939 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.c 2026-02-13T06:43:08.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:169: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-13T06:43:12.041 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.d 2026-02-13T06:43:12.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:170: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-13T06:43:12.143 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.e 2026-02-13T06:43:12.148 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:172: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-13T06:43:12.149 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:43:12.149 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:43:12.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:43:12.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:43:12.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:43:12.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:43:12.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:173: TEST_5_mons_checks: wait_for_quorum 60 4 2026-02-13T06:43:12.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T06:43:12.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-13T06:43:12.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T06:43:12.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-13T06:43:12.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T06:43:12.257 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T06:43:12.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770965052 2026-02-13T06:43:12.258 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:12.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964992 -lt 1770965052 ]] 2026-02-13T06:43:12.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:12.259 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:12.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:12.361 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:12.362 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:12.362 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:12.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:12.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:12.364 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:12.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964992 -lt 1770965052 ]] 2026-02-13T06:43:12.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:12.365 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:15.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:15.468 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.469 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.469 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:15.471 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:15.471 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:15.471 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:15.472 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964995 -lt 1770965052 ]] 2026-02-13T06:43:15.472 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:15.472 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:15.576 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:15.576 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.577 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:15.577 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:15.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:15.579 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:15.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964995 -lt 1770965052 ]] 2026-02-13T06:43:15.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:15.580 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:15.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:15.681 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.681 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.681 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:15.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:15.684 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:15.684 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:15.685 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964995 -lt 1770965052 ]] 2026-02-13T06:43:15.685 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:15.685 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:15.787 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:15.787 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:15.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:15.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:15.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:15.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964995 -lt 1770965052 ]] 2026-02-13T06:43:15.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:15.792 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:15.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:15.894 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.895 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:15.895 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:15.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:15.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:15.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:15.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964995 -lt 1770965052 ]] 2026-02-13T06:43:15.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:15.899 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.001 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.107 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.107 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.111 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.212 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.212 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.213 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.213 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.215 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.215 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.216 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.317 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.317 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.318 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.318 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.321 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.322 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.421 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.422 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.422 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.424 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.425 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.425 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.526 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.526 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.529 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.529 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.529 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.530 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.630 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.631 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.631 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.737 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.737 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.738 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.739 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.740 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.740 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.741 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.741 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.741 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.841 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.842 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.842 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.844 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.845 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:16.945 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:16.945 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:16.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:16.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:16.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:16.948 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:16.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964996 -lt 1770965052 ]] 2026-02-13T06:43:16.949 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:16.949 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:17.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:17.049 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.050 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:17.050 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:17.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:17.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:17.053 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964997 -lt 1770965052 ]] 2026-02-13T06:43:17.053 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:17.054 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:17.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:17.157 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:17.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:17.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:17.160 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:17.161 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964997 -lt 1770965052 ]] 2026-02-13T06:43:17.161 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:17.162 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:17.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:17.262 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.263 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.263 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:17.265 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:17.265 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:17.265 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:17.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964997 -lt 1770965052 ]] 2026-02-13T06:43:17.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:17.266 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:17.368 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:17.368 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.369 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.369 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:17.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:17.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:17.371 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:17.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964997 -lt 1770965052 ]] 2026-02-13T06:43:17.372 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:17.372 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:17.472 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:17.472 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.473 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.473 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:17.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:17.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:17.475 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:17.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964997 -lt 1770965052 ]] 2026-02-13T06:43:17.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:17.476 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:17.577 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:17.577 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.578 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.578 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:17.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:17.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:17.580 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:17.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964997 -lt 1770965052 ]] 2026-02-13T06:43:17.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:17.581 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:17.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:17.686 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.686 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:17.687 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:17.689 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:17.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770964997 -lt 1770965052 ]] 2026-02-13T06:43:17.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:17.690 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:20.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:20.791 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:20.792 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:20.792 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:20.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:20.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:20.795 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:20.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965000 -lt 1770965052 ]] 2026-02-13T06:43:20.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T06:43:20.796 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:27.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:27.501 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:27.501 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:27.502 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T06:43:27.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T06:43:27.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T06:43:27.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T06:43:27.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T06:43:27.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T06:43:27.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:175: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-13T06:43:27.599 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-13T06:43:27.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:176: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-13T06:43:27.701 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (c,d,e) after stopping [b] 2026-02-13T06:43:27.706 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:177: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-13T06:43:27.801 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,d,e) after stopping [c] 2026-02-13T06:43:27.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:178: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-13T06:43:27.901 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,c,e) after stopping [d] 2026-02-13T06:43:27.907 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:179: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-13T06:43:28.001 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-13T06:43:28.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:181: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-13T06:43:28.103 INFO:tasks.workunit.client.0.trial040.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T06:43:28.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:183: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-13T06:43:28.206 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.a 2026-02-13T06:43:28.212 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:184: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-13T06:43:28.310 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.b 2026-02-13T06:43:28.316 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:185: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-13T06:43:28.413 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.c 2026-02-13T06:43:28.419 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:186: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-13T06:43:28.517 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.d 2026-02-13T06:43:28.523 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:187: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-13T06:43:31.621 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.e 2026-02-13T06:43:31.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:189: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.e 2026-02-13T06:43:31.628 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:43:31.628 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:43:31.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:43:31.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:43:31.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:43:31.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:43:31.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:190: TEST_5_mons_checks: wait_for_quorum 60 3 2026-02-13T06:43:31.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T06:43:31.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-13T06:43:31.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T06:43:31.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-13T06:43:31.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T06:43:31.735 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T06:43:31.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770965071 2026-02-13T06:43:31.736 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:31.736 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965011 -lt 1770965071 ]] 2026-02-13T06:43:31.736 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:31.737 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:31.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:31.839 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:31.840 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:31.840 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:31.842 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:31.842 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:31.842 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:31.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965011 -lt 1770965071 ]] 2026-02-13T06:43:31.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:31.843 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:34.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:34.944 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:34.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:34.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:34.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:34.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:34.947 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:34.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965014 -lt 1770965071 ]] 2026-02-13T06:43:34.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:34.948 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:35.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:35.052 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:35.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:35.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:35.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:35.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:35.054 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:35.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965015 -lt 1770965071 ]] 2026-02-13T06:43:35.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:35.056 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:35.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:35.155 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:35.156 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:35.156 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:35.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:35.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:35.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:35.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965015 -lt 1770965071 ]] 2026-02-13T06:43:35.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:35.159 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:35.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:35.259 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:35.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:35.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:35.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:35.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:35.263 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:35.263 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965015 -lt 1770965071 ]] 2026-02-13T06:43:35.263 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:35.264 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:35.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:35.363 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:35.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:35.364 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:35.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:35.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:35.365 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:35.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965015 -lt 1770965071 ]] 2026-02-13T06:43:35.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:35.367 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:38.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:38.470 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.471 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.471 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:38.473 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:38.473 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:38.473 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:38.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965018 -lt 1770965071 ]] 2026-02-13T06:43:38.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:38.474 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:38.576 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:38.576 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.577 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:38.577 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:38.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:38.579 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:38.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965018 -lt 1770965071 ]] 2026-02-13T06:43:38.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:38.580 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:38.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:38.683 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.684 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.684 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:38.685 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:38.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:38.686 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:38.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965018 -lt 1770965071 ]] 2026-02-13T06:43:38.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:38.687 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:38.787 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:38.787 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:38.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:38.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:38.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:38.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965018 -lt 1770965071 ]] 2026-02-13T06:43:38.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:38.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:38.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:38.895 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.896 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:38.896 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:38.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:38.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:38.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:38.899 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965018 -lt 1770965071 ]] 2026-02-13T06:43:38.899 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:38.899 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:39.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:39.001 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:39.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:39.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:39.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:39.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:39.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:39.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965019 -lt 1770965071 ]] 2026-02-13T06:43:39.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:39.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:39.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:39.111 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:39.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:39.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:39.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:39.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:39.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:39.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965019 -lt 1770965071 ]] 2026-02-13T06:43:39.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:39.116 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:39.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:39.218 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:39.219 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:39.219 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:39.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:39.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:39.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:39.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965019 -lt 1770965071 ]] 2026-02-13T06:43:39.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:39.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:39.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:39.321 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:39.322 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:39.322 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:39.323 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T06:43:39.324 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T06:43:39.324 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T06:43:39.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770965019 -lt 1770965071 ]] 2026-02-13T06:43:39.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T06:43:39.325 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T06:43:46.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T06:43:46.831 INFO:tasks.workunit.client.0.trial040.stderr:{"election_epoch":14,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:46.832 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":14,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"ebfb88b7-81a8-44f2-a0a1-6b7fd3bfcd10","modified":"2026-02-13T06:42:51.861968Z","created":"2026-02-13T06:42:51.861968Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"crush_location":"{}"}]}}' 2026-02-13T06:43:46.833 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T06:43:46.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T06:43:46.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T06:43:46.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T06:43:46.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T06:43:46.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T06:43:46.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:192: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-13T06:43:46.929 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-13T06:43:46.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:193: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-13T06:43:50.012 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-13T06:43:50.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:194: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-13T06:43:50.090 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-13T06:43:50.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:195: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-13T06:43:50.170 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-13T06:43:50.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:196: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-13T06:43:50.270 INFO:tasks.workunit.client.0.trial040.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-13T06:43:50.276 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:198: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-13T06:43:50.354 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-13T06:43:50.356 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:200: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-13T06:43:50.451 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.a 2026-02-13T06:43:50.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:201: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-13T06:43:56.534 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: removing mon.b would break quorum 2026-02-13T06:43:56.536 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:202: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-13T06:43:56.614 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-13T06:43:56.616 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:203: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-13T06:43:56.693 INFO:tasks.workunit.client.0.trial040.stderr:Error EBUSY: removing mon.d would break quorum 2026-02-13T06:43:56.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:204: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-13T06:43:56.791 INFO:tasks.workunit.client.0.trial040.stderr:safe to remove mon.e 2026-02-13T06:43:56.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T06:43:56.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:43:56.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:43:56.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:43:56.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:43:56.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:43:56.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:43:56.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T06:43:56.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:43:56.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:43:56.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:43:56.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:43:56.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:43:56.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:43:56.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:43:56.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:43:56.910 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:43:56.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:43:56.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:43:56.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:43:56.912 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:43:56.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:43:56.913 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:43:56.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:43:56.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:43:56.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:43:56.914 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:43:56.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:43:56.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:43:56.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:43:56.926 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:43:56.926 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:43:56.926 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:43:56.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:43:56.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:43:56.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:43:56.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-13T06:43:56.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/ok-to-stop 0 2026-02-13T06:43:56.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T06:43:56.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T06:43:56.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T06:43:56.928 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:43:56.929 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:43:56.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:43:56.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:43:56.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:43:56.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:43:56.929 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:43:56.930 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:43:56.930 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:43:56.931 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:43:56.931 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:43:56.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:43:56.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:43:56.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:43:56.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:43:56.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:43:56.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:43:56.934 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T06:43:56.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T06:43:56.936 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:43:56.936 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:43:56.936 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.43705 2026-02-13T06:43:56.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.43705 2026-02-13T06:43:56.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:43:56.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:43:56.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-13T06:43:56.938 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T06:43:56.938 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T06:43:56.965 INFO:tasks.workunit:Running workunit misc/rados-striper.sh... 2026-02-13T06:43:56.966 DEBUG:teuthology.orchestra.run.trial040:workunit test misc/rados-striper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh 2026-02-13T06:43:57.028 INFO:tasks.workunit.client.0.trial040.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T06:43:57.031 INFO:tasks.workunit.client.0.trial040.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T06:43:57.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/rados-striper 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:20: run: local dir=td/rados-striper 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:21: run: shift 2026-02-13T06:43:57.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: export CEPH_MON=127.0.0.1:7116 2026-02-13T06:43:57.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: CEPH_MON=127.0.0.1:7116 2026-02-13T06:43:57.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:24: run: export CEPH_ARGS 2026-02-13T06:43:57.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: uuidgen 2026-02-13T06:43:57.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: CEPH_ARGS+='--fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none ' 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7116 ' 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:29: run: setup td/rados-striper 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/rados-striper 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/rados-striper 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:43:57.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:43:57.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:43:57.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:43:57.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:43:57.037 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:43:57.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:43:57.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:43:57.037 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:43:57.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:43:57.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:43:57.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:43:57.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:43:57.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:43:57.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:43:57.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-13T06:43:57.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:43:57.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:43:57.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:43:57.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59100 2026-02-13T06:43:57.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:43:57.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:43:57.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/rados-striper 2026-02-13T06:43:57.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:43:57.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:43:57.043 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:43:57.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59100 2026-02-13T06:43:57.043 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/rados-striper 1' TERM HUP INT 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:32: run: run_mon td/rados-striper a 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/rados-striper 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/rados-striper/a 2026-02-13T06:43:57.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/rados-striper/a --run-dir=td/rados-striper 2026-02-13T06:43:57.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:43:57.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:43:57.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:43:57.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:43:57.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:43:57.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:43:57.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59100/$cluster-$name.asok' 2026-02-13T06:43:57.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/rados-striper/a '--log-file=td/rados-striper/$name.log' '--admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' --mon-cluster-log-file=td/rados-striper/log --run-dir=td/rados-striper '--pid-file=td/rados-striper/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:43:57.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:43:57.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:43:57.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:43:57.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:43:57.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:43:57.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:43:57.093 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:43:57.093 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:43:57.093 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:43:57.094 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:43:57.094 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:43:57.094 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:43:57.094 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59100/ceph-mon.a.asok 2026-02-13T06:43:57.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:43:57.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59100/ceph-mon.a.asok config get fsid 2026-02-13T06:43:57.141 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59100/ceph-mon.a.asok 2026-02-13T06:43:57.142 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:43:57.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59100/ceph-mon.a.asok config get mon_host 2026-02-13T06:43:57.188 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:33: run: run_osd td/rados-striper 0 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/0 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:43:57.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/0' 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/0/journal' 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59100/$cluster-$name.asok' 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' 2026-02-13T06:43:57.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/0 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:43:57.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ed156c84-1735-4cc9-954a-e8bd382840de 2026-02-13T06:43:57.192 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ed156c84-1735-4cc9-954a-e8bd382840de' 2026-02-13T06:43:57.192 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 ed156c84-1735-4cc9-954a-e8bd382840de 2026-02-13T06:43:57.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:43:57.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAtyI5paPnPCxAACC+LKRR8M+6Z2jCOF0HoLA== 2026-02-13T06:43:57.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAtyI5paPnPCxAACC+LKRR8M+6Z2jCOF0HoLA=="}' 2026-02-13T06:43:57.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ed156c84-1735-4cc9-954a-e8bd382840de -i td/rados-striper/0/new.json 2026-02-13T06:43:57.301 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:43:57.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/0/new.json 2026-02-13T06:43:57.308 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAtyI5paPnPCxAACC+LKRR8M+6Z2jCOF0HoLA== --osd-uuid ed156c84-1735-4cc9-954a-e8bd382840de 2026-02-13T06:43:57.324 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:57.322+0000 7f3453613900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:57.325 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:57.323+0000 7f3453613900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:57.326 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:57.324+0000 7f3453613900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:57.327 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:57.324+0000 7f3453613900 -1 bdev(0x560b4806ac00 td/rados-striper/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:43:57.327 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:57.324+0000 7f3453613900 -1 bluestore(td/rados-striper/0) _read_fsid unparsable uuid 2026-02-13T06:43:57.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/0/keyring 2026-02-13T06:43:57.660 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:43:57.660 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:43:57.661 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:43:57.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:43:57.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:43:57.764 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:43:57.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:43:57.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:43:57.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:43:57.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:43:57.780 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:57.777+0000 7fe9c83c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:57.781 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:57.779+0000 7fe9c83c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:57.782 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:57.779+0000 7fe9c83c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:57.859 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:43:57.859 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:43:57.859 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:43:57.859 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:43:57.859 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:43:57.860 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:43:57.860 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:43:57.860 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:43:57.861 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:43:57.861 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:43:57.928 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:57.926+0000 7fe9c83c8900 -1 Falling back to public interface 2026-02-13T06:43:57.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:43:58.201 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:58.198+0000 7fe9c83c8900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:43:58.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:43:58.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:43:58.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:43:58.957 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:43:58.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:43:58.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:43:59.052 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/507512608,v1:127.0.0.1:6801/507512608] [v2:127.0.0.1:6802/507512608,v1:127.0.0.1:6803/507512608] exists,up ed156c84-1735-4cc9-954a-e8bd382840de 2026-02-13T06:43:59.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:43:59.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:43:59.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:34: run: run_osd td/rados-striper 1 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/1 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:43:59.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/1' 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/1/journal' 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:43:59.059 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59100/$cluster-$name.asok' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:43:59.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:43:59.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:43:59.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:43:59.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:43:59.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/1 2026-02-13T06:43:59.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:43:59.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f42fd8da-1a90-480b-afea-d32e93ee4ca7 2026-02-13T06:43:59.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 f42fd8da-1a90-480b-afea-d32e93ee4ca7' 2026-02-13T06:43:59.061 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 f42fd8da-1a90-480b-afea-d32e93ee4ca7 2026-02-13T06:43:59.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:43:59.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAvyI5pgVbiAxAAZnMcPv8ZFv8LyfbBWlIv8w== 2026-02-13T06:43:59.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAvyI5pgVbiAxAAZnMcPv8ZFv8LyfbBWlIv8w=="}' 2026-02-13T06:43:59.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f42fd8da-1a90-480b-afea-d32e93ee4ca7 -i td/rados-striper/1/new.json 2026-02-13T06:43:59.170 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:43:59.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/1/new.json 2026-02-13T06:43:59.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAvyI5pgVbiAxAAZnMcPv8ZFv8LyfbBWlIv8w== --osd-uuid f42fd8da-1a90-480b-afea-d32e93ee4ca7 2026-02-13T06:43:59.194 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:59.192+0000 7f716fb29900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:59.195 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:59.193+0000 7f716fb29900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:59.196 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:59.194+0000 7f716fb29900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:59.197 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:59.194+0000 7f716fb29900 -1 bdev(0x563e4cf74c00 td/rados-striper/1/block) open stat got: (1) Operation not permitted 2026-02-13T06:43:59.197 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:59.195+0000 7f716fb29900 -1 bluestore(td/rados-striper/1) _read_fsid unparsable uuid 2026-02-13T06:43:59.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/1/keyring 2026-02-13T06:43:59.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:43:59.516 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T06:43:59.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T06:43:59.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:43:59.616 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T06:43:59.617 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T06:43:59.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:43:59.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:43:59.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:43:59.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:43:59.633 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:59.630+0000 7fb7f5b97900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:59.634 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:59.632+0000 7fb7f5b97900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:59.635 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:59.633+0000 7fb7f5b97900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:43:59.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T06:43:59.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:43:59.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T06:43:59.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:43:59.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:43:59.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:43:59.717 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:43:59.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:43:59.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:43:59.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:43:59.811 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:43:59.841 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:43:59.838+0000 7fb7f5b97900 -1 Falling back to public interface 2026-02-13T06:44:00.088 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:00.086+0000 7fb7f5b97900 -1 osd.1 0 log_to_monitors true 2026-02-13T06:44:00.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:00.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:00.813 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:00.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:00.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:00.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:44:00.914 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/2358208382,v1:127.0.0.1:6809/2358208382] [v2:127.0.0.1:6810/2358208382,v1:127.0.0.1:6811/2358208382] exists,up f42fd8da-1a90-480b-afea-d32e93ee4ca7 2026-02-13T06:44:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:35: run: run_osd td/rados-striper 2 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/2 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/2' 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/2/journal' 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:00.919 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59100/$cluster-$name.asok' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:00.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:00.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/2 2026-02-13T06:44:00.921 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:00.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f7ef2797-36b6-49b0-9601-e4a14c3e906e 2026-02-13T06:44:00.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f7ef2797-36b6-49b0-9601-e4a14c3e906e' 2026-02-13T06:44:00.921 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 f7ef2797-36b6-49b0-9601-e4a14c3e906e 2026-02-13T06:44:00.921 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:00.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAwyI5pdodENxAABh0UmmDlacK2Nu48cAmY8Q== 2026-02-13T06:44:00.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAwyI5pdodENxAABh0UmmDlacK2Nu48cAmY8Q=="}' 2026-02-13T06:44:00.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f7ef2797-36b6-49b0-9601-e4a14c3e906e -i td/rados-striper/2/new.json 2026-02-13T06:44:01.029 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T06:44:01.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/2/new.json 2026-02-13T06:44:01.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAwyI5pdodENxAABh0UmmDlacK2Nu48cAmY8Q== --osd-uuid f7ef2797-36b6-49b0-9601-e4a14c3e906e 2026-02-13T06:44:01.052 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.049+0000 7f5d8698e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:01.053 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.051+0000 7f5d8698e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:01.054 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.052+0000 7f5d8698e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:01.054 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.052+0000 7f5d8698e900 -1 bdev(0x55bf353cec00 td/rados-striper/2/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:01.055 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.052+0000 7f5d8698e900 -1 bluestore(td/rados-striper/2) _read_fsid unparsable uuid 2026-02-13T06:44:01.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/2/keyring 2026-02-13T06:44:01.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:01.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T06:44:01.382 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T06:44:01.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:01.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T06:44:01.485 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T06:44:01.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=fa8b9c50-8977-4dbe-8bce-3ae2d6c648f1 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.59100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:01.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:01.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:01.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:01.501 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.499+0000 7fd0fbf75900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:01.503 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.500+0000 7fd0fbf75900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:01.504 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.501+0000 7fd0fbf75900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:01.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T06:44:01.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:01.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T06:44:01.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:01.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:01.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:01.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:01.582 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:01.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:01.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T06:44:01.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:01.704 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.702+0000 7fd0fbf75900 -1 Falling back to public interface 2026-02-13T06:44:01.969 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:01.967+0000 7fd0fbf75900 -1 osd.2 0 log_to_monitors true 2026-02-13T06:44:02.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:02.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:02.677 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:02.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:02.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:02.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T06:44:02.771 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/409488443,v1:127.0.0.1:6817/409488443] [v2:127.0.0.1:6818/409488443,v1:127.0.0.1:6819/409488443] exists,up f7ef2797-36b6-49b0-9601-e4a14c3e906e 2026-02-13T06:44:02.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:02.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:02.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:02.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:36: run: create_rbd_pool 2026-02-13T06:44:02.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:44:02.861 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:44:02.867 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:44:02.867 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:44:03.000 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:44:03.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:44:04.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:44:04.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:39: run: dd if=/dev/urandom of=td/rados-striper/toyfile bs=1234 count=1 2026-02-13T06:44:04.337 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records in 2026-02-13T06:44:04.337 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records out 2026-02-13T06:44:04.337 INFO:tasks.workunit.client.0.trial040.stderr:1234 bytes (1.2 kB, 1.2 KiB) copied, 4.9963e-05 s, 24.7 MB/s 2026-02-13T06:44:04.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:42: run: rados --pool rbd --striper put toyfile td/rados-striper/toyfile 2026-02-13T06:44:04.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: rados --pool rbd --striper stat toyfile 2026-02-13T06:44:04.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: cut -d , -f 2 2026-02-13T06:44:04.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-13T06:44:04.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: cut -d , -f 2 2026-02-13T06:44:04.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:47: run: echo ' size 1234' 2026-02-13T06:44:04.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:48: run: diff -w td/rados-striper/stripedStat td/rados-striper/refstat 2026-02-13T06:44:04.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:49: run: diff -w td/rados-striper/stat td/rados-striper/refstat 2026-02-13T06:44:04.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:50: run: rados --pool rbd stat toyfile 2026-02-13T06:44:04.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:51: run: grep -q 'No such file or directory' td/rados-striper/staterror 2026-02-13T06:44:04.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:54: run: rados --pool rbd --striper get toyfile td/rados-striper/stripedGroup 2026-02-13T06:44:04.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:55: run: diff -w td/rados-striper/toyfile td/rados-striper/stripedGroup 2026-02-13T06:44:04.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:56: run: rados --pool rbd get toyfile.0000000000000000 td/rados-striper/nonSTripedGroup 2026-02-13T06:44:04.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:57: run: diff -w td/rados-striper/toyfile td/rados-striper/nonSTripedGroup 2026-02-13T06:44:04.448 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:60: run: rados --pool rbd --striper truncate toyfile 12 2026-02-13T06:44:04.467 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: rados --pool rbd --striper stat toyfile 2026-02-13T06:44:04.467 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: cut -d , -f 2 2026-02-13T06:44:04.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-13T06:44:04.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: cut -d , -f 2 2026-02-13T06:44:04.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:63: run: echo ' size 12' 2026-02-13T06:44:04.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:64: run: diff -w td/rados-striper/stripedStat td/rados-striper/reftrunc 2026-02-13T06:44:04.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:65: run: diff -w td/rados-striper/stat td/rados-striper/reftrunc 2026-02-13T06:44:04.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:69: run: rados --pool rbd --striper setxattr toyfile somexattr somevalue 2026-02-13T06:44:04.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:70: run: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-13T06:44:04.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:71: run: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-13T06:44:04.553 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:72: run: echo somevalue 2026-02-13T06:44:04.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:73: run: diff -w td/rados-striper/xattrvalue td/rados-striper/refvalue 2026-02-13T06:44:04.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:74: run: diff -w td/rados-striper/xattrvalue2 td/rados-striper/refvalue 2026-02-13T06:44:04.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:75: run: rados --pool rbd --striper listxattr toyfile 2026-02-13T06:44:04.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:76: run: echo somexattr 2026-02-13T06:44:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:77: run: diff -w td/rados-striper/xattrlist td/rados-striper/reflist 2026-02-13T06:44:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: rados --pool rbd listxattr toyfile.0000000000000000 2026-02-13T06:44:04.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: grep -v striper 2026-02-13T06:44:04.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:79: run: diff -w td/rados-striper/xattrlist2 td/rados-striper/reflist 2026-02-13T06:44:04.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:80: run: rados --pool rbd --striper rmxattr toyfile somexattr 2026-02-13T06:44:04.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:82: run: local 'attr_not_found_str=No data available' 2026-02-13T06:44:04.608 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: uname 2026-02-13T06:44:04.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: '[' Linux = FreeBSD ']' 2026-02-13T06:44:04.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:85: run: expect_failure td/rados-striper 'No data available' rados --pool rbd --striper getxattr toyfile somexattr 2026-02-13T06:44:04.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-13T06:44:04.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T06:44:04.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-13T06:44:04.610 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T06:44:04.610 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T06:44:04.610 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-13T06:44:04.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T06:44:04.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T06:44:04.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-13T06:44:04.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T06:44:04.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:87: run: expect_failure td/rados-striper 'No data available' rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-13T06:44:04.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-13T06:44:04.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T06:44:04.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-13T06:44:04.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T06:44:04.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T06:44:04.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-13T06:44:04.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T06:44:04.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T06:44:04.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-13T06:44:04.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T06:44:04.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:91: run: rados --pool rbd --striper rm toyfile 2026-02-13T06:44:04.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:92: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd --striper stat toyfile 2026-02-13T06:44:04.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-13T06:44:04.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T06:44:04.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-13T06:44:04.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T06:44:04.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T06:44:04.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper stat toyfile 2026-02-13T06:44:04.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T06:44:04.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T06:44:04.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-13T06:44:04.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T06:44:04.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:94: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd stat toyfile.0000000000000000 2026-02-13T06:44:04.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-13T06:44:04.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T06:44:04.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-13T06:44:04.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T06:44:04.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T06:44:04.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd stat toyfile.0000000000000000 2026-02-13T06:44:04.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T06:44:04.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T06:44:04.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-13T06:44:04.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T06:44:04.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:98: run: teardown td/rados-striper 2026-02-13T06:44:04.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-13T06:44:04.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:04.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-13T06:44:04.695 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:04.695 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:04.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:04.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:04.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:04.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:04.802 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:04.802 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:04.803 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:04.804 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:04.804 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:04.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:04.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:04.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:04.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:04.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:04.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:04.808 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:04.808 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-13T06:44:04.813 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:04.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:04.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:44:04.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59100 2026-02-13T06:44:04.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:04.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:04.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-13T06:44:04.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/rados-striper 0 2026-02-13T06:44:04.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-13T06:44:04.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T06:44:04.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-13T06:44:04.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:04.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:04.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:04.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:04.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:04.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:04.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:04.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:04.818 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:04.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:04.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:04.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:04.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:04.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:04.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:04.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:04.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:04.822 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T06:44:04.822 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-13T06:44:04.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:04.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:04.824 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59100 2026-02-13T06:44:04.824 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59100 2026-02-13T06:44:04.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:04.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:04.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-13T06:44:04.825 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T06:44:04.826 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T06:44:04.889 INFO:tasks.workunit:Running workunit misc/test-ceph-helpers.sh... 2026-02-13T06:44:04.890 DEBUG:teuthology.orchestra.run.trial040:workunit test misc/test-ceph-helpers.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh 2026-02-13T06:44:04.953 INFO:tasks.workunit.client.0.trial040.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T06:44:04.956 INFO:tasks.workunit.client.0.trial040.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T06:44:04.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2415: run_tests: export PATH=./bin:.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:44:04.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2415: run_tests: PATH=./bin:.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T06:44:04.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2417: run_tests: export CEPH_MON=127.0.0.1:7109 2026-02-13T06:44:04.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2417: run_tests: CEPH_MON=127.0.0.1:7109 2026-02-13T06:44:04.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: export CEPH_ARGS 2026-02-13T06:44:04.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2419: run_tests: uuidgen 2026-02-13T06:44:04.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2419: run_tests: CEPH_ARGS+=' --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none ' 2026-02-13T06:44:04.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: CEPH_ARGS+='--mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:04.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2421: run_tests: export CEPH_CONF=/dev/null 2026-02-13T06:44:04.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2421: run_tests: CEPH_CONF=/dev/null 2026-02-13T06:44:04.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: run_tests: set 2026-02-13T06:44:04.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: run_tests: sed -n -e 's/^\(test_[0-9a-z_]*\) .*/\1/p' 2026-02-13T06:44:04.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: run_tests: local 'funcs=test_activate_osd 2026-02-13T06:44:04.960 INFO:tasks.workunit.client.0.trial040.stderr:test_activate_osd_after_mark_down 2026-02-13T06:44:04.960 INFO:tasks.workunit.client.0.trial040.stderr:test_activate_osd_skip_benchmark 2026-02-13T06:44:04.960 INFO:tasks.workunit.client.0.trial040.stderr:test_destroy_osd 2026-02-13T06:44:04.960 INFO:tasks.workunit.client.0.trial040.stderr:test_display_logs 2026-02-13T06:44:04.960 INFO:tasks.workunit.client.0.trial040.stderr:test_erasure_code_plugin_exists 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_expect_failure 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_file_with_random_data 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_flush_pg_stats 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_get_config 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_get_is_making_recovery_progress 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_get_last_scrub_stamp 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_get_not_primary 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_get_num_active_clean 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_get_num_active_or_peered 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_get_num_pgs 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_get_op_scheduler 2026-02-13T06:44:04.961 INFO:tasks.workunit.client.0.trial040.stderr:test_get_osd_id_used_by_pgs 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_get_osds 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_get_pg 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_get_primary 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_get_timeout_delays 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_is_clean 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_kill_daemon 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_kill_daemons 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_objectstore_tool 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_pg_schedule_scrub 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_pg_scrub 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_repair 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_run_in_background 2026-02-13T06:44:04.962 INFO:tasks.workunit.client.0.trial040.stderr:test_run_mon 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_run_osd 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_set_config 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_setup 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_teardown 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_wait_background 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_wait_for_clean 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_wait_for_health_ok 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_wait_for_osd 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_wait_for_peered 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_wait_for_scrub 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:test_wait_osd_id_used_by_pgs' 2026-02-13T06:44:04.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: run_tests: local dir=td/ceph-helpers 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_activate_osd td/ceph-helpers 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: test_activate_osd: local dir=td/ceph-helpers 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: test_activate_osd: setup td/ceph-helpers 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:04.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:04.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:04.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:04.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:04.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:04.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:04.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:04.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:04.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:04.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:04.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:04.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:04.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:04.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:04.969 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:04.969 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:04.969 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:04.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:04.973 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:04.973 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T06:44:04.973 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T06:44:04.973 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:04.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:04.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:881: test_activate_osd: run_mon td/ceph-helpers a 2026-02-13T06:44:04.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:04.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:04.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:04.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:04.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:04.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T06:44:04.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:04.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:04.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:04.991 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:04.991 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:04.991 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:04.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:04.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:44:05.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:05.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:05.019 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:05.020 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:05.020 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:05.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:05.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:05.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:05.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:05.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:05.065 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:05.065 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:05.065 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:05.066 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:05.066 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:05.066 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:05.066 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:05.066 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:05.066 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:05.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: test_activate_osd: run_mgr td/ceph-helpers x 2026-02-13T06:44:05.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:05.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:05.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:05.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:05.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:05.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:05.213 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:05.213 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:05.213 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:05.213 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:05.213 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:05.213 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:05.213 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:05.214 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:05.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:05.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:884: test_activate_osd: run_osd td/ceph-helpers 0 2026-02-13T06:44:05.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:05.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:05.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:05.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:05.235 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:05.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=34dd6288-382f-45ed-aa30-24b386e2e409 2026-02-13T06:44:05.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 34dd6288-382f-45ed-aa30-24b386e2e409' 2026-02-13T06:44:05.238 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 34dd6288-382f-45ed-aa30-24b386e2e409 2026-02-13T06:44:05.238 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:05.246 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA1yI5p77mVDhAANBoL7NENrzroHZA/krjOWw== 2026-02-13T06:44:05.246 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA1yI5p77mVDhAANBoL7NENrzroHZA/krjOWw=="}' 2026-02-13T06:44:05.246 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 34dd6288-382f-45ed-aa30-24b386e2e409 -i td/ceph-helpers/0/new.json 2026-02-13T06:44:05.349 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:05.356 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:05.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA1yI5p77mVDhAANBoL7NENrzroHZA/krjOWw== --osd-uuid 34dd6288-382f-45ed-aa30-24b386e2e409 2026-02-13T06:44:05.375 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:05.373+0000 7f74bec1f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:05.377 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:05.374+0000 7f74bec1f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:05.378 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:05.375+0000 7f74bec1f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:05.378 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:05.376+0000 7f74bec1f900 -1 bdev(0x55f20fbacc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:05.378 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:05.376+0000 7f74bec1f900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:05.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:05.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:05.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:05.814 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:05.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:05.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:05.921 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:05.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:05.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:05.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:05.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:05.937 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:05.935+0000 7fc57f80c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:05.939 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:05.936+0000 7fc57f80c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:05.940 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:05.937+0000 7fc57f80c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:06.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:06.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:06.155 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:06.152+0000 7fc57f80c900 -1 Falling back to public interface 2026-02-13T06:44:06.464 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:06.461+0000 7fc57f80c900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:07.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:07.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:07.119 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:07.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:07.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:07.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:07.220 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2044116768,v1:127.0.0.1:6801/2044116768] [v2:127.0.0.1:6802/2044116768,v1:127.0.0.1:6803/2044116768] exists,up 34dd6288-382f-45ed-aa30-24b386e2e409 2026-02-13T06:44:07.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:07.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:07.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:07.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: get_asok_path osd.0 2026-02-13T06:44:07.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:44:07.222 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:44:07.222 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:07.222 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:07.222 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:07.223 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:44:07.223 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: CEPH_ARGS= 2026-02-13T06:44:07.223 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T06:44:07.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: local 'backfills={"osd_max_backfills":"1"}' 2026-02-13T06:44:07.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: echo '{"osd_max_backfills":"1"}' 2026-02-13T06:44:07.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: grep --quiet osd_max_backfills 2026-02-13T06:44:07.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:889: test_activate_osd: kill_daemons td/ceph-helpers TERM osd 2026-02-13T06:44:07.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:07.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:07.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:07.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:07.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:07.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:07.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:891: test_activate_osd: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-13T06:44:07.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T06:44:07.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T06:44:07.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:07.584 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:07.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:07.586 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:07.586 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:07.586 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T06:44:07.586 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-13T06:44:07.586 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:07.586 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T06:44:07.586 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:07.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-13T06:44:07.587 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-13T06:44:07.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T06:44:07.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T06:44:07.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T06:44:07.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T06:44:07.601 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:07.598+0000 7f8ea2627900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:07.602 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:07.600+0000 7f8ea2627900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:07.603 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:07.601+0000 7f8ea2627900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:07.770 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:07.767+0000 7f8ea2627900 -1 Falling back to public interface 2026-02-13T06:44:07.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T06:44:07.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:07.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:07.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:07.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:07.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:07.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:07.817 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:07.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:07.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:08.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:08.157 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:08.155+0000 7f8ea2627900 -1 osd.0 5 log_to_monitors true 2026-02-13T06:44:09.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:09.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:09.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:09.045 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:09.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:09.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:09.269 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/2993543644,v1:127.0.0.1:6801/2993543644] [v2:127.0.0.1:6802/2993543644,v1:127.0.0.1:6803/2993543644] exists,up 34dd6288-382f-45ed-aa30-24b386e2e409 2026-02-13T06:44:09.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:09.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: get_asok_path osd.0 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: CEPH_ARGS= 2026-02-13T06:44:09.271 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T06:44:09.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: local 'backfills={"osd_max_backfills":"20"}' 2026-02-13T06:44:09.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:894: test_activate_osd: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-13T06:44:09.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:896: test_activate_osd: teardown td/ceph-helpers 2026-02-13T06:44:09.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:09.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:09.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:09.320 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:09.321 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:09.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:09.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:09.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:09.426 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:09.426 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:09.427 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:09.427 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:09.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:09.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:09.429 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:09.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:09.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:09.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:09.431 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:09.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:09.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:09.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:09.436 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:09.436 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:09.436 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:09.436 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:09.438 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:09.438 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:09.438 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:09.438 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_activate_osd_after_mark_down td/ceph-helpers 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:900: test_activate_osd_after_mark_down: local dir=td/ceph-helpers 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:902: test_activate_osd_after_mark_down: setup td/ceph-helpers 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:09.439 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:09.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:09.440 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:09.440 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:09.440 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:09.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:09.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:09.441 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:09.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:09.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:09.442 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:09.443 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:09.443 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:09.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:09.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:09.445 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:09.445 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:09.445 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:09.445 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:09.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:09.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:09.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:09.447 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:09.447 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:09.447 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:09.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:09.448 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:09.449 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:09.449 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:09.449 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:09.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:904: test_activate_osd_after_mark_down: run_mon td/ceph-helpers a 2026-02-13T06:44:09.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:09.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:09.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:09.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:09.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:09.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T06:44:09.469 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:09.469 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:09.469 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:09.469 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:09.469 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:09.470 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:09.470 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:09.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:44:09.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:09.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:09.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:09.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:09.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:09.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:09.498 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:09.498 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:09.498 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:09.499 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:09.500 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:09.500 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:09.500 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:09.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:09.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:09.545 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:09.545 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:09.545 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:09.545 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:09.545 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:09.546 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:09.546 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:09.546 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:09.547 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:09.547 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:09.547 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:09.547 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:09.547 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:09.547 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:09.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:905: test_activate_osd_after_mark_down: run_mgr td/ceph-helpers x 2026-02-13T06:44:09.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:09.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:09.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:09.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:09.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:09.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:09.692 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:09.692 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:09.692 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:09.692 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:09.693 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:09.693 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:09.693 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:09.693 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:09.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:09.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:907: test_activate_osd_after_mark_down: run_osd td/ceph-helpers 0 2026-02-13T06:44:09.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:09.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:09.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:09.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:09.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:09.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:09.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:09.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:09.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:09.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:09.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:09.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:09.717 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:09.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9e7a45fe-98f8-4152-9e9d-b4669d25faa4 2026-02-13T06:44:09.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9e7a45fe-98f8-4152-9e9d-b4669d25faa4' 2026-02-13T06:44:09.718 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 9e7a45fe-98f8-4152-9e9d-b4669d25faa4 2026-02-13T06:44:09.718 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:09.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA5yI5pfMYwKxAAXHa5vYUXN9Ujh4v2PogDTg== 2026-02-13T06:44:09.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA5yI5pfMYwKxAAXHa5vYUXN9Ujh4v2PogDTg=="}' 2026-02-13T06:44:09.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9e7a45fe-98f8-4152-9e9d-b4669d25faa4 -i td/ceph-helpers/0/new.json 2026-02-13T06:44:09.832 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:09.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:09.840 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA5yI5pfMYwKxAAXHa5vYUXN9Ujh4v2PogDTg== --osd-uuid 9e7a45fe-98f8-4152-9e9d-b4669d25faa4 2026-02-13T06:44:09.858 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:09.856+0000 7f521c222900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:09.860 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:09.857+0000 7f521c222900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:09.861 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:09.858+0000 7f521c222900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:09.861 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:09.859+0000 7f521c222900 -1 bdev(0x55a192d32c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:09.861 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:09.859+0000 7f521c222900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:10.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:10.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:10.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:10.262 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:10.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:10.370 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:10.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:10.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:10.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:10.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:10.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:10.387 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:10.385+0000 7f3227f90900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:10.389 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:10.386+0000 7f3227f90900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:10.390 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:10.387+0000 7f3227f90900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:10.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:10.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:10.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:10.471 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:10.471 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:10.471 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:10.471 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:10.471 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:10.472 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:10.472 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:10.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:10.627 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:10.624+0000 7f3227f90900 -1 Falling back to public interface 2026-02-13T06:44:10.949 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:10.946+0000 7f3227f90900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:11.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:11.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:11.572 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:11.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:11.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:11.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:11.674 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/276126921,v1:127.0.0.1:6801/276126921] [v2:127.0.0.1:6802/276126921,v1:127.0.0.1:6803/276126921] exists,up 9e7a45fe-98f8-4152-9e9d-b4669d25faa4 2026-02-13T06:44:11.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:11.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:11.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:11.676 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-13T06:44:11.676 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:44:11.676 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:44:11.676 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:11.676 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:11.676 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:11.676 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:44:11.677 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-13T06:44:11.677 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T06:44:11.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"1"}' 2026-02-13T06:44:11.727 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: echo '{"osd_max_backfills":"1"}' 2026-02-13T06:44:11.727 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: grep --quiet osd_max_backfills 2026-02-13T06:44:11.728 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:912: test_activate_osd_after_mark_down: kill_daemons td/ceph-helpers TERM osd 2026-02-13T06:44:11.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:11.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:11.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:11.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:11.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:12.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:12.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:913: test_activate_osd_after_mark_down: ceph osd down 0 2026-02-13T06:44:12.259 INFO:tasks.workunit.client.0.trial040.stderr:osd.0 is already down. 2026-02-13T06:44:12.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:914: test_activate_osd_after_mark_down: wait_for_osd down 0 2026-02-13T06:44:12.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-13T06:44:12.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:12.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:12.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:12.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:12.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:12.268 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:12.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:12.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-13T06:44:12.493 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/276126921,v1:127.0.0.1:6801/276126921] [v2:127.0.0.1:6802/276126921,v1:127.0.0.1:6803/276126921] exists 9e7a45fe-98f8-4152-9e9d-b4669d25faa4 2026-02-13T06:44:12.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:12.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:12.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: test_activate_osd_after_mark_down: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:12.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:12.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:12.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:12.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:12.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:12.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:12.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:12.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:12.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T06:44:12.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-13T06:44:12.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:12.500 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:12.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T06:44:12.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-13T06:44:12.501 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-13T06:44:12.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T06:44:12.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T06:44:12.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T06:44:12.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T06:44:12.512 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:12.511+0000 7ff975897900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:12.514 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:12.512+0000 7ff975897900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:12.515 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:12.513+0000 7ff975897900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:12.672 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:12.670+0000 7ff975897900 -1 Falling back to public interface 2026-02-13T06:44:12.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T06:44:12.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:12.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:12.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:12.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:12.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:12.721 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:12.721 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:12.721 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:12.721 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:12.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:13.066 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:13.064+0000 7ff975897900 -1 osd.0 5 log_to_monitors true 2026-02-13T06:44:13.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:13.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:13.950 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:13.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:13.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:13.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:14.176 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/3984060625,v1:127.0.0.1:6801/3984060625] [v2:127.0.0.1:6802/3984060625,v1:127.0.0.1:6803/3984060625] exists,up 9e7a45fe-98f8-4152-9e9d-b4669d25faa4 2026-02-13T06:44:14.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:14.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:14.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:14.177 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-13T06:44:14.177 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:44:14.177 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:44:14.177 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:14.177 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:14.177 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:14.177 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:44:14.177 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-13T06:44:14.177 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T06:44:14.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"20"}' 2026-02-13T06:44:14.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: test_activate_osd_after_mark_down: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-13T06:44:14.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: test_activate_osd_after_mark_down: teardown td/ceph-helpers 2026-02-13T06:44:14.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:14.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:14.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:14.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:14.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:14.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:14.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:14.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:14.334 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:14.334 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:14.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:14.335 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:14.336 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:14.336 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:14.337 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:14.338 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:14.338 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:14.338 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:14.338 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:14.339 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:14.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:14.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:14.343 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:14.344 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:14.344 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:14.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_activate_osd_skip_benchmark td/ceph-helpers 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: test_activate_osd_skip_benchmark: local dir=td/ceph-helpers 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:927: test_activate_osd_skip_benchmark: setup td/ceph-helpers 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:14.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:14.347 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:14.347 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:14.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:14.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:14.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:14.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:14.348 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:14.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:14.348 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:14.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:14.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:14.349 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:14.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:14.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:14.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:14.351 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:14.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:14.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:14.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:14.353 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:14.353 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:14.353 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:14.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:14.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:14.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:14.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:14.355 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:14.356 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:14.356 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:14.356 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:14.356 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: test_activate_osd_skip_benchmark: run_mon td/ceph-helpers a 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:14.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T06:44:14.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:14.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:14.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:14.378 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:14.378 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:14.378 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:14.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:14.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:44:14.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:14.406 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:14.406 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:14.406 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:14.406 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:14.406 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:14.407 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:14.407 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:14.407 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:14.408 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:14.408 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:14.408 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:14.408 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:14.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:14.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:14.454 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:14.454 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:14.454 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:14.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:14.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:930: test_activate_osd_skip_benchmark: run_mgr td/ceph-helpers x 2026-02-13T06:44:14.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:14.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:14.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:14.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:14.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:14.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:14.597 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:14.598 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:14.598 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:14.598 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:14.598 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:14.598 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:14.598 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:14.599 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:14.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:14.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:933: test_activate_osd_skip_benchmark: run_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-13T06:44:14.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:14.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:14.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:14.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:14.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:14.621 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:14.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:14.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:14.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:14.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:14.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-13T06:44:14.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:14.623 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:14.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=04301d22-2a92-4003-8631-8639fee7c969 2026-02-13T06:44:14.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 04301d22-2a92-4003-8631-8639fee7c969' 2026-02-13T06:44:14.623 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 04301d22-2a92-4003-8631-8639fee7c969 2026-02-13T06:44:14.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:14.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA+yI5pZWeQJRAAY8b6EVbhyynLZAcuShGe7A== 2026-02-13T06:44:14.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA+yI5pZWeQJRAAY8b6EVbhyynLZAcuShGe7A=="}' 2026-02-13T06:44:14.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 04301d22-2a92-4003-8631-8639fee7c969 -i td/ceph-helpers/0/new.json 2026-02-13T06:44:14.735 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:14.741 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:14.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true --mkfs --key AQA+yI5pZWeQJRAAY8b6EVbhyynLZAcuShGe7A== --osd-uuid 04301d22-2a92-4003-8631-8639fee7c969 2026-02-13T06:44:14.762 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:14.760+0000 7fccd39b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:14.763 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:14.762+0000 7fccd39b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:14.764 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:14.763+0000 7fccd39b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:14.765 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:14.763+0000 7fccd39b0900 -1 bdev(0x558d66788c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:14.765 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:14.763+0000 7fccd39b0900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:15.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:15.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:15.228 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:15.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:15.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:15.333 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:15.334 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:15.334 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-13T06:44:15.334 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:15.334 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:15.334 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:15.351 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:15.350+0000 7f5d21462900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:15.352 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:15.351+0000 7f5d21462900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:15.353 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:15.352+0000 7f5d21462900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:15.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:15.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:15.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:15.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:15.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:15.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:15.435 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:15.435 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:15.435 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:15.435 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:15.538 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:15.592 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:15.591+0000 7f5d21462900 -1 Falling back to public interface 2026-02-13T06:44:16.033 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:16.031+0000 7f5d21462900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:16.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:16.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:16.540 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:16.540 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:16.540 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:16.540 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:16.643 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3906337634,v1:127.0.0.1:6801/3906337634] [v2:127.0.0.1:6802/3906337634,v1:127.0.0.1:6803/3906337634] exists,up 04301d22-2a92-4003-8631-8639fee7c969 2026-02-13T06:44:16.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:16.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:16.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:16.644 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-13T06:44:16.644 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:44:16.644 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:44:16.645 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:16.645 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:16.645 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:16.645 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:44:16.645 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-13T06:44:16.645 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-13T06:44:16.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: local 'max_iops_hdd_def={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-13T06:44:16.694 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-13T06:44:16.695 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:44:16.695 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:44:16.695 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:16.695 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:16.695 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:16.695 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:44:16.695 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-13T06:44:16.695 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-13T06:44:16.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: local 'max_iops_ssd_def={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-13T06:44:16.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:940: test_activate_osd_skip_benchmark: kill_daemons td/ceph-helpers TERM osd 2026-02-13T06:44:16.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:16.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:16.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:16.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:16.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:17.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:17.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:941: test_activate_osd_skip_benchmark: ceph osd down 0 2026-02-13T06:44:17.278 INFO:tasks.workunit.client.0.trial040.stderr:osd.0 is already down. 2026-02-13T06:44:17.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:942: test_activate_osd_skip_benchmark: wait_for_osd down 0 2026-02-13T06:44:17.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-13T06:44:17.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:17.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:17.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:17.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:17.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:17.288 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:17.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:17.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-13T06:44:17.512 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/3906337634,v1:127.0.0.1:6801/3906337634] [v2:127.0.0.1:6802/3906337634,v1:127.0.0.1:6803/3906337634] exists 04301d22-2a92-4003-8631-8639fee7c969 2026-02-13T06:44:17.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:946: test_activate_osd_skip_benchmark: activate_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:17.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:17.517 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:17.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T06:44:17.519 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:17.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-13T06:44:17.520 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-13T06:44:17.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T06:44:17.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T06:44:17.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T06:44:17.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T06:44:17.533 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:17.531+0000 7f31a9f40900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:17.534 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:17.533+0000 7f31a9f40900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:17.535 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:17.534+0000 7f31a9f40900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:17.733 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:17.732+0000 7f31a9f40900 -1 Falling back to public interface 2026-02-13T06:44:17.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T06:44:17.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:17.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:17.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:17.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:17.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:17.743 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:17.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:17.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:17.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:17.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:18.078 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:18.077+0000 7f31a9f40900 -1 osd.0 6 log_to_monitors true 2026-02-13T06:44:18.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:18.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:18.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:18.970 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:18.971 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:18.971 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:19.197 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [5,6) [v2:127.0.0.1:6800/1150582311,v1:127.0.0.1:6801/1150582311] [v2:127.0.0.1:6802/1150582311,v1:127.0.0.1:6803/1150582311] exists,up 04301d22-2a92-4003-8631-8639fee7c969 2026-02-13T06:44:19.198 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:19.198 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:19.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:19.199 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-13T06:44:19.199 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:44:19.199 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:44:19.200 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:19.200 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.200 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.200 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:44:19.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-13T06:44:19.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-13T06:44:19.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: local 'max_iops_hdd_after_boot={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-13T06:44:19.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-13T06:44:19.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:44:19.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:44:19.250 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:19.250 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.250 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.250 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:44:19.251 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-13T06:44:19.251 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-13T06:44:19.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: local 'max_iops_ssd_after_boot={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-13T06:44:19.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:953: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' = '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-13T06:44:19.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:954: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' = '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-13T06:44:19.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:956: test_activate_osd_skip_benchmark: teardown td/ceph-helpers 2026-02-13T06:44:19.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:19.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:19.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:19.302 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:19.302 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:19.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:19.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:19.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:19.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:19.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:19.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:19.410 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:19.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:19.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:19.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:19.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:19.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:19.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:19.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:19.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:19.415 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:19.415 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:19.418 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:19.418 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.418 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.418 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:19.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:19.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:19.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:19.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_destroy_osd td/ceph-helpers 2026-02-13T06:44:19.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:787: test_destroy_osd: local dir=td/ceph-helpers 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:789: test_destroy_osd: setup td/ceph-helpers 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:19.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:19.422 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:19.422 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:19.422 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:19.423 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:19.423 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:19.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:19.424 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:19.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:19.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:19.425 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:19.425 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:19.425 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:19.426 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:19.426 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:19.427 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:19.428 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.428 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:19.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:19.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:19.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:19.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:19.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:19.431 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:790: test_destroy_osd: run_mon td/ceph-helpers a 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:19.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T06:44:19.452 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:19.452 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:19.453 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:19.453 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:19.453 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.453 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.453 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:19.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:44:19.479 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:19.480 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.481 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:19.482 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:19.482 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:19.525 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:19.525 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:19.525 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:19.525 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:19.525 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:19.525 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:19.525 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:19.526 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:19.526 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:19.527 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.527 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.527 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:19.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:19.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:19.570 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:791: test_destroy_osd: run_mgr td/ceph-helpers x 2026-02-13T06:44:19.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:19.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:19.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:19.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:19.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:19.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:19.672 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:19.672 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:19.672 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:19.672 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:19.672 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.672 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.672 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:19.673 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:19.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:19.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:792: test_destroy_osd: run_osd td/ceph-helpers 0 2026-02-13T06:44:19.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:19.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:19.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:19.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:19.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:19.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:19.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:19.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:19.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:19.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:19.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:19.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:19.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:19.695 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:19.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ce3248ff-03c6-4674-b159-67f804f72f3d 2026-02-13T06:44:19.695 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 ce3248ff-03c6-4674-b159-67f804f72f3d 2026-02-13T06:44:19.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ce3248ff-03c6-4674-b159-67f804f72f3d' 2026-02-13T06:44:19.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:19.707 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBDyI5pfDQUKhAARwU5nsc54YwdQLVNwy7QYw== 2026-02-13T06:44:19.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBDyI5pfDQUKhAARwU5nsc54YwdQLVNwy7QYw=="}' 2026-02-13T06:44:19.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ce3248ff-03c6-4674-b159-67f804f72f3d -i td/ceph-helpers/0/new.json 2026-02-13T06:44:19.814 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:19.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:19.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBDyI5pfDQUKhAARwU5nsc54YwdQLVNwy7QYw== --osd-uuid ce3248ff-03c6-4674-b159-67f804f72f3d 2026-02-13T06:44:19.840 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:19.838+0000 7f82b3422900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:19.841 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:19.840+0000 7f82b3422900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:19.843 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:19.841+0000 7f82b3422900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:19.843 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:19.841+0000 7f82b3422900 -1 bdev(0x560fd3e88c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:19.843 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:19.841+0000 7f82b3422900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:20.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:20.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:20.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:20.258 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:20.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:20.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:20.364 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:20.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:20.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:20.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:20.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:20.381 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:20.379+0000 7f80dcd49900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:20.382 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:20.381+0000 7f80dcd49900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:20.383 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:20.382+0000 7f80dcd49900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:20.467 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:20.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:20.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:20.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:20.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:20.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:20.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:20.468 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:20.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:20.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:20.570 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:20.578 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:20.577+0000 7f80dcd49900 -1 Falling back to public interface 2026-02-13T06:44:20.980 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:20.979+0000 7f80dcd49900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:21.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:21.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:21.572 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:21.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:21.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:21.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:21.673 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/719755106,v1:127.0.0.1:6801/719755106] [v2:127.0.0.1:6802/719755106,v1:127.0.0.1:6803/719755106] exists,up ce3248ff-03c6-4674-b159-67f804f72f3d 2026-02-13T06:44:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:793: test_destroy_osd: destroy_osd td/ceph-helpers 0 2026-02-13T06:44:21.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:775: destroy_osd: local dir=td/ceph-helpers 2026-02-13T06:44:21.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:776: destroy_osd: local id=0 2026-02-13T06:44:21.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:778: destroy_osd: ceph osd out osd.0 2026-02-13T06:44:21.822 INFO:tasks.workunit.client.0.trial040.stderr:osd.0 is already out. 2026-02-13T06:44:21.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:779: destroy_osd: kill_daemons td/ceph-helpers TERM osd.0 2026-02-13T06:44:21.829 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:21.829 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:21.829 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:21.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:21.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:22.136 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:22.136 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:780: destroy_osd: ceph osd down osd.0 2026-02-13T06:44:22.360 INFO:tasks.workunit.client.0.trial040.stderr:osd.0 is already down. 2026-02-13T06:44:22.368 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:781: destroy_osd: ceph osd purge osd.0 --yes-i-really-mean-it 2026-02-13T06:44:22.593 INFO:tasks.workunit.client.0.trial040.stderr:osd.0 does not exist 2026-02-13T06:44:22.601 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:782: destroy_osd: teardown td/ceph-helpers/0 2026-02-13T06:44:22.601 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers/0 2026-02-13T06:44:22.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:22.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers/0 KILL 2026-02-13T06:44:22.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:22.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:22.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:22.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:22.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:22.603 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:22.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:22.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:22.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:22.605 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:22.605 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:22.606 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:22.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:22.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:22.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:22.607 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:22.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:22.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:22.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers/0 2026-02-13T06:44:22.610 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:22.610 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:22.610 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:22.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:22.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:22.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:22.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:783: destroy_osd: rm -fr td/ceph-helpers/0 2026-02-13T06:44:22.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: ceph osd dump 2026-02-13T06:44:22.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: grep 'osd. ' 2026-02-13T06:44:22.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:795: test_destroy_osd: teardown td/ceph-helpers 2026-02-13T06:44:22.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:22.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:22.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:22.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:22.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:22.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:22.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:22.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:22.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:22.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:22.946 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:22.946 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:22.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:22.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:22.948 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:22.949 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:22.949 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:22.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:22.950 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:22.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:22.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:22.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:22.954 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:22.954 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:22.954 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:22.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:22.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:22.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:22.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:22.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_display_logs td/ceph-helpers 2026-02-13T06:44:22.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2158: test_display_logs: local dir=td/ceph-helpers 2026-02-13T06:44:22.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2160: test_display_logs: setup td/ceph-helpers 2026-02-13T06:44:22.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:22.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:22.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:22.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:22.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:22.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:22.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:22.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:22.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:22.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:22.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:22.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:22.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:22.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:22.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:22.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:22.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:22.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:22.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:22.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:22.961 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:22.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:22.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:22.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:22.963 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:22.963 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:22.963 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:22.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:22.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:22.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:22.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:22.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:22.966 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:22.966 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:22.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:22.966 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:22.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:22.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:22.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:22.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2161: test_display_logs: run_mon td/ceph-helpers a 2026-02-13T06:44:22.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:22.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:22.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:22.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:22.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:22.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T06:44:22.988 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:22.988 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:22.988 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:22.988 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:22.988 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:22.988 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:22.988 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:22.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:44:23.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:23.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:23.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:23.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:23.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:23.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:23.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:23.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:23.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:23.018 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:23.018 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.018 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:23.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:23.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:23.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:23.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:23.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:23.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:23.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:23.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:23.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:23.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:23.064 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:23.064 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.064 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:23.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:23.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:23.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2162: test_display_logs: kill_daemons td/ceph-helpers 2026-02-13T06:44:23.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:23.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:23.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:23.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:23.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:23.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:23.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2163: test_display_logs: display_logs td/ceph-helpers 2026-02-13T06:44:23.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2148: display_logs: local dir=td/ceph-helpers 2026-02-13T06:44:23.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: display_logs: find td/ceph-helpers -maxdepth 1 -name '*.log' 2026-02-13T06:44:23.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-13T06:44:23.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2152: display_logs: echo '======================= td/ceph-helpers/mon.a.log' 2026-02-13T06:44:23.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2153: display_logs: cat td/ceph-helpers/mon.a.log 2026-02-13T06:44:23.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-13T06:44:23.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2164: test_display_logs: grep --quiet mon.a.log td/ceph-helpers/log.out 2026-02-13T06:44:23.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2165: test_display_logs: teardown td/ceph-helpers 2026-02-13T06:44:23.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:23.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:23.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:23.219 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:23.219 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:23.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:23.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:23.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:23.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:23.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:23.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:23.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:23.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:23.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:23.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:23.223 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:23.223 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:23.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:23.224 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:23.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:23.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:23.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:23.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:23.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:23.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:23.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:23.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:23.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_erasure_code_plugin_exists td/ceph-helpers 2026-02-13T06:44:23.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: test_erasure_code_plugin_exists: local dir=td/ceph-helpers 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: test_erasure_code_plugin_exists: setup td/ceph-helpers 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:23.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:23.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:23.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:23.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:23.232 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:23.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:23.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:23.232 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:23.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:23.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:23.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:23.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:23.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:23.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:23.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:23.236 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:23.237 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.237 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:23.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:23.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:23.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:23.239 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:23.239 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.239 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:23.240 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: test_erasure_code_plugin_exists: run_mon td/ceph-helpers a 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:23.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T06:44:23.261 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:23.261 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:23.262 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:23.262 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:23.262 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.262 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.262 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:23.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:44:23.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:23.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:23.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:23.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:23.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:23.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:23.291 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:23.291 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:23.291 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:23.291 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:23.291 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.291 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.291 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:23.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:23.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:23.335 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:23.335 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:23.335 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:23.335 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:23.335 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:23.337 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:23.337 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:23.337 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:23.337 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:23.337 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.337 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.337 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:23.337 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:23.337 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:23.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: test_erasure_code_plugin_exists: run_mgr td/ceph-helpers x 2026-02-13T06:44:23.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:23.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:23.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:23.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:23.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:23.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:23.485 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:23.485 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:23.485 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:23.486 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:23.486 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.486 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.486 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:23.486 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:23.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:23.503 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2134: test_erasure_code_plugin_exists: erasure_code_plugin_exists jerasure 2026-02-13T06:44:23.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=jerasure 2026-02-13T06:44:23.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-13T06:44:23.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-13T06:44:23.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-13T06:44:23.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-13T06:44:23.504 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-13T06:44:23.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='jerasure.*No such file' 2026-02-13T06:44:23.505 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=jerasure 2026-02-13T06:44:23.614 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-13T06:44:23.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-13T06:44:23.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-13T06:44:23.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-13T06:44:23.781 INFO:tasks.workunit.client.0.trial040.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-13T06:44:23.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-13T06:44:23.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2135: test_erasure_code_plugin_exists: erasure_code_plugin_exists FAKE 2026-02-13T06:44:23.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=FAKE 2026-02-13T06:44:23.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-13T06:44:23.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-13T06:44:23.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-13T06:44:23.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-13T06:44:23.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-13T06:44:23.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='FAKE.*No such file' 2026-02-13T06:44:23.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=FAKE 2026-02-13T06:44:23.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s='Error EIO: load dlopen(/usr/lib64/ceph/erasure-code/libec_FAKE.so): /usr/lib64/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory' 2026-02-13T06:44:23.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=5 2026-02-13T06:44:23.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 5 -eq 0 ']' 2026-02-13T06:44:23.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: echo Error EIO: load 'dlopen(/usr/lib64/ceph/erasure-code/libec_FAKE.so):' /usr/lib64/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory 2026-02-13T06:44:23.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: grep --quiet 'FAKE.*No such file' 2026-02-13T06:44:23.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2119: erasure_code_plugin_exists: status=1 2026-02-13T06:44:23.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 1 2026-02-13T06:44:23.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2136: test_erasure_code_plugin_exists: teardown td/ceph-helpers 2026-02-13T06:44:23.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:23.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:23.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:23.879 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:23.879 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:23.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:23.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:23.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:23.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:23.984 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:23.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:23.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:23.986 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:23.987 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:23.987 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:23.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:23.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:23.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:23.989 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:23.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:23.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:23.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:23.992 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:23.992 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:23.992 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:23.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:23.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:23.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:23.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:23.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_expect_failure td/ceph-helpers 2026-02-13T06:44:23.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: test_expect_failure: local dir=td/ceph-helpers 2026-02-13T06:44:23.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2039: test_expect_failure: setup td/ceph-helpers 2026-02-13T06:44:23.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:23.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:23.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:23.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:23.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:23.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:23.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:23.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:23.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:23.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:23.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:23.996 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:23.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:23.996 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:23.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:23.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:23.997 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:23.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:23.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:23.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:23.999 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:23.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:24.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:24.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:24.001 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:24.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:24.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:24.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:24.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:24.003 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:24.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:24.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:24.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:24.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:24.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:24.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2040: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 1' 2026-02-13T06:44:24.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-13T06:44:24.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T06:44:24.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-13T06:44:24.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T06:44:24.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T06:44:24.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 1' 2026-02-13T06:44:24.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T06:44:24.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T06:44:24.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-13T06:44:24.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T06:44:24.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2042: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 0' 2026-02-13T06:44:24.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-13T06:44:24.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T06:44:24.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-13T06:44:24.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T06:44:24.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T06:44:24.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 0' 2026-02-13T06:44:24.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2023: expect_failure: success=true 2026-02-13T06:44:24.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: true 2026-02-13T06:44:24.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-13T06:44:24.011 INFO:tasks.workunit.client.0.trial040.stderr:FAIL 2026-02-13T06:44:24.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-13T06:44:24.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2043: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-13T06:44:24.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2045: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo UNEXPECTED ; exit 1' 2026-02-13T06:44:24.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-13T06:44:24.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T06:44:24.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-13T06:44:24.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T06:44:24.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T06:44:24.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo UNEXPECTED ; exit 1' 2026-02-13T06:44:24.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T06:44:24.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T06:44:24.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-13T06:44:24.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-13T06:44:24.014 INFO:tasks.workunit.client.0.trial040.stderr:UNEXPECTED 2026-02-13T06:44:24.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-13T06:44:24.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2046: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-13T06:44:24.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2047: test_expect_failure: teardown td/ceph-helpers 2026-02-13T06:44:24.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:24.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:24.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:24.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:24.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:24.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:24.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:24.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:24.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:24.017 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:24.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:24.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:24.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:24.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:24.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:24.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:24.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:24.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:24.020 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:24.021 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:24.021 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:24.021 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:24.023 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:24.023 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.023 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_file_with_random_data td/ceph-helpers 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2351: test_file_with_random_data: local dir=td/ceph-helpers 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: setup td/ceph-helpers 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:24.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:24.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:24.026 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:24.026 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:24.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:24.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:24.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:24.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:24.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:24.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:24.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:24.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:24.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:24.028 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:24.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:24.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:24.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:24.029 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:24.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:24.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:24.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:24.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:24.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:24.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:24.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:24.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:24.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:24.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:24.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:24.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:24.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:24.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:24.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2354: test_file_with_random_data: file_with_random_data 4000 2026-02-13T06:44:24.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=4000 2026-02-13T06:44:24.036 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-13T06:44:24.037 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.m5Fc9HiXdy 2026-02-13T06:44:24.037 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.m5Fc9HiXdy bs=4000 count=1 2026-02-13T06:44:24.038 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records in 2026-02-13T06:44:24.038 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records out 2026-02-13T06:44:24.038 INFO:tasks.workunit.client.0.trial040.stderr:4000 bytes (4.0 kB, 3.9 KiB) copied, 5.2638e-05 s, 76.0 MB/s 2026-02-13T06:44:24.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.m5Fc9HiXdy 2026-02-13T06:44:24.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2354: test_file_with_random_data: local file=/tmp/tmp.m5Fc9HiXdy 2026-02-13T06:44:24.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: test -f /tmp/tmp.m5Fc9HiXdy 2026-02-13T06:44:24.039 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2356: test_file_with_random_data: stat -c %s /tmp/tmp.m5Fc9HiXdy 2026-02-13T06:44:24.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2356: test_file_with_random_data: test 4000 = 4000 2026-02-13T06:44:24.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2357: test_file_with_random_data: rm /tmp/tmp.m5Fc9HiXdy 2026-02-13T06:44:24.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2358: test_file_with_random_data: teardown td/ceph-helpers 2026-02-13T06:44:24.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:24.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:24.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:24.041 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:24.041 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:24.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:24.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:24.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:24.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:24.042 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:24.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:24.043 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:24.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:24.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:24.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:24.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:24.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:24.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:24.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:24.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:24.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:24.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:24.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:24.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:24.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:24.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_flush_pg_stats td/ceph-helpers 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2281: test_flush_pg_stats: local dir=td/ceph-helpers 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2283: test_flush_pg_stats: setup td/ceph-helpers 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:24.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:24.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:24.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:24.052 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:24.053 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:24.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:24.053 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:24.053 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:24.054 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:24.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:24.054 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:24.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:24.055 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:24.055 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:24.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:24.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:24.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:24.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:24.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:24.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:24.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:24.060 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:24.060 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.060 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:24.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:24.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:24.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:24.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:24.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: test_flush_pg_stats: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:24.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:24.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:24.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:24.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:24.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:24.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:24.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:24.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:24.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:24.083 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:24.083 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.083 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:24.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:24.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:24.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:24.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:24.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:24.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:24.114 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:24.156 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:24.157 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:24.157 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:24.157 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:24.157 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:24.158 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:24.158 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:24.158 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:24.158 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:24.159 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.159 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.159 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:24.159 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:24.159 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:24.201 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: test_flush_pg_stats: run_mgr td/ceph-helpers x 2026-02-13T06:44:24.201 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:24.201 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:24.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:24.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:24.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:24.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:24.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:24.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:24.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:24.304 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:24.304 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.304 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.304 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:24.304 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:24.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:24.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: test_flush_pg_stats: run_osd td/ceph-helpers 0 2026-02-13T06:44:24.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:24.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:24.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:24.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:24.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:24.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:24.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:24.326 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:24.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:24.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:24.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:24.328 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:24.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5f98f0b5-05dd-4885-b476-f21d56ec2732 2026-02-13T06:44:24.328 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 5f98f0b5-05dd-4885-b476-f21d56ec2732 2026-02-13T06:44:24.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5f98f0b5-05dd-4885-b476-f21d56ec2732' 2026-02-13T06:44:24.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:24.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBIyI5pLfcGFBAAZrNacHo0H/rqPWufXZCOjQ== 2026-02-13T06:44:24.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBIyI5pLfcGFBAAZrNacHo0H/rqPWufXZCOjQ=="}' 2026-02-13T06:44:24.338 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5f98f0b5-05dd-4885-b476-f21d56ec2732 -i td/ceph-helpers/0/new.json 2026-02-13T06:44:24.444 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:24.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:24.452 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBIyI5pLfcGFBAAZrNacHo0H/rqPWufXZCOjQ== --osd-uuid 5f98f0b5-05dd-4885-b476-f21d56ec2732 2026-02-13T06:44:24.471 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:24.470+0000 7f2db7853900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:24.473 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:24.471+0000 7f2db7853900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:24.474 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:24.473+0000 7f2db7853900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:24.474 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:24.473+0000 7f2db7853900 -1 bdev(0x5635831ecc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:24.474 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:24.473+0000 7f2db7853900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:24.888 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:24.888 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:24.888 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:24.889 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:24.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:24.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:24.993 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:24.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:24.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:24.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:24.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:25.010 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:25.009+0000 7fa674e0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:25.012 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:25.010+0000 7fa674e0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:25.013 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:25.011+0000 7fa674e0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:25.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:25.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:25.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:25.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:25.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:25.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:25.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:25.092 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:25.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:25.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:25.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:25.199 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:25.197+0000 7fa674e0e900 -1 Falling back to public interface 2026-02-13T06:44:25.611 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:25.609+0000 7fa674e0e900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:26.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:26.196 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:26.196 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:26.196 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:26.197 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:26.197 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:26.301 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2793121899,v1:127.0.0.1:6801/2793121899] [v2:127.0.0.1:6802/2793121899,v1:127.0.0.1:6803/2793121899] exists,up 5f98f0b5-05dd-4885-b476-f21d56ec2732 2026-02-13T06:44:26.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:26.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:26.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:26.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: test_flush_pg_stats: create_rbd_pool 2026-02-13T06:44:26.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:44:26.396 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:44:26.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:44:26.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:44:26.680 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:44:26.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:44:27.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:44:27.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: test_flush_pg_stats: rados -p rbd put obj /etc/group 2026-02-13T06:44:27.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: test_flush_pg_stats: flush_pg_stats 2026-02-13T06:44:27.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:44:27.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:44:28.223 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:44:28.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:44:28.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:44:28.224 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:44:28.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:44:28.290 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:44:28.290 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:44:28.290 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:44:28.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:28.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:44:28.291 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:44:28.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:28.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:44:28.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:44:28.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:44:28.293 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:44:28.293 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:44:28.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:44:28.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: local 'jq_filter=.pools | .[] | select(.name == "rbd") | .stats' 2026-02-13T06:44:28.521 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: ceph df detail --format=json 2026-02-13T06:44:28.521 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored' 2026-02-13T06:44:28.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: stored=8139 2026-02-13T06:44:28.835 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2292: test_flush_pg_stats: ceph df detail --format=json 2026-02-13T06:44:28.835 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2292: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored_raw' 2026-02-13T06:44:29.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2292: test_flush_pg_stats: stored_raw=8139 2026-02-13T06:44:29.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2293: test_flush_pg_stats: test 8139 -gt 0 2026-02-13T06:44:29.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: test_flush_pg_stats: test 8139 == 8139 2026-02-13T06:44:29.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: test_flush_pg_stats: teardown td/ceph-helpers 2026-02-13T06:44:29.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:29.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:29.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:29.181 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:29.181 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:29.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:29.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:29.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:29.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:29.287 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:29.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:29.288 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:29.290 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:29.290 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:29.290 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:29.291 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:29.291 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:29.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:29.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:29.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:29.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:29.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:29.297 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:29.297 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:29.297 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:29.297 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:29.299 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:29.299 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:29.299 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:29.299 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_config td/ceph-helpers 2026-02-13T06:44:29.299 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: test_get_config: local dir=td/ceph-helpers 2026-02-13T06:44:29.299 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: test_get_config: setup td/ceph-helpers 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:29.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:29.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:29.301 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:29.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:29.301 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:29.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:29.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:29.302 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:29.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:29.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:29.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:29.304 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:29.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:29.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:29.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:29.306 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:29.306 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:29.306 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:29.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:29.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:29.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:29.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:29.308 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:29.309 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:29.309 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:29.309 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:29.310 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:29.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:29.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:29.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:29.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1131: test_get_config: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:29.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:29.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:29.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:29.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:29.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:29.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:29.332 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:29.332 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:29.332 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:29.333 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:29.333 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:29.333 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:29.333 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:29.333 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:29.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:29.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:29.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:29.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:29.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:29.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:29.361 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:29.362 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:29.362 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:29.362 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:29.362 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:29.362 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:29.362 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:29.362 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:29.362 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:29.407 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:29.407 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:29.408 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:29.409 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:29.409 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:29.454 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: get_config mon a osd_pool_default_size 2026-02-13T06:44:29.454 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:29.454 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:29.454 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_pool_default_size 2026-02-13T06:44:29.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_pool_default_size 2026-02-13T06:44:29.456 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:29.456 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:29.456 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:29.456 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:29.456 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:29.456 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:29.456 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:29.456 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:29.456 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get osd_pool_default_size 2026-02-13T06:44:29.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: test 1 = 1 2026-02-13T06:44:29.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1133: test_get_config: run_mgr td/ceph-helpers x 2026-02-13T06:44:29.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:29.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:29.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:29.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:29.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:29.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:29.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:29.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:29.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:29.603 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:29.603 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:29.603 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:29.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:29.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:29.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:29.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: test_get_config: run_osd td/ceph-helpers 0 --osd_max_scrubs=3 2026-02-13T06:44:29.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:29.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:29.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:29.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:29.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:29.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:29.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:29.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:29.626 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:29.627 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:29.627 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:29.627 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:29.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:29.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:29.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:29.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:29.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:29.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:29.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:29.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:29.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:29.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:29.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:29.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_max_scrubs=3 2026-02-13T06:44:29.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:29.628 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:29.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=be82dade-bcb1-4d29-8090-3be1adae2285 2026-02-13T06:44:29.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 be82dade-bcb1-4d29-8090-3be1adae2285' 2026-02-13T06:44:29.629 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 be82dade-bcb1-4d29-8090-3be1adae2285 2026-02-13T06:44:29.630 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:29.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBNyI5pw4ziJRAAKrz1ouWfIfPvmZWTa/Nn7A== 2026-02-13T06:44:29.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBNyI5pw4ziJRAAKrz1ouWfIfPvmZWTa/Nn7A=="}' 2026-02-13T06:44:29.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new be82dade-bcb1-4d29-8090-3be1adae2285 -i td/ceph-helpers/0/new.json 2026-02-13T06:44:29.739 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:29.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:29.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 --mkfs --key AQBNyI5pw4ziJRAAKrz1ouWfIfPvmZWTa/Nn7A== --osd-uuid be82dade-bcb1-4d29-8090-3be1adae2285 2026-02-13T06:44:29.764 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:29.762+0000 7f659e6e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:29.765 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:29.764+0000 7f659e6e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:29.766 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:29.765+0000 7f659e6e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:29.767 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:29.765+0000 7f659e6e0900 -1 bdev(0x56040f4e4c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:29.767 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:29.765+0000 7f659e6e0900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:30.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:30.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:30.215 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:30.215 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:30.215 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:30.322 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:30.322 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:30.323 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 2026-02-13T06:44:30.323 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:30.323 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:30.323 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:30.340 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:30.339+0000 7f9aac010900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:30.342 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:30.340+0000 7f9aac010900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:30.343 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:30.341+0000 7f9aac010900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:30.425 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:30.425 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:30.425 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:30.425 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:30.425 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:30.426 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:30.426 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:30.427 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:30.427 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:30.427 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:30.527 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:30.553 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:30.551+0000 7f9aac010900 -1 Falling back to public interface 2026-02-13T06:44:30.857 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:30.856+0000 7f9aac010900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:31.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:31.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:31.529 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:31.529 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:31.529 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:31.529 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:31.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:32.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:32.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:32.633 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T06:44:32.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-13T06:44:32.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:32.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:32.862 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/886327520,v1:127.0.0.1:6801/886327520] [v2:127.0.0.1:6802/886327520,v1:127.0.0.1:6803/886327520] exists,up be82dade-bcb1-4d29-8090-3be1adae2285 2026-02-13T06:44:32.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: get_config osd 0 osd_max_scrubs 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_max_scrubs 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_max_scrubs 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:44:32.864 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:32.865 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:32.865 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:32.865 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:44:32.865 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:32.865 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_max_scrubs 2026-02-13T06:44:32.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: test 3 = 3 2026-02-13T06:44:32.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1136: test_get_config: teardown td/ceph-helpers 2026-02-13T06:44:32.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:32.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:32.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:32.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:32.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:32.912 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:32.912 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:32.912 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:33.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:33.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:33.020 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:33.020 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:33.021 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:33.022 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:33.022 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:33.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:33.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:33.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:33.023 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:33.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:33.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:33.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:33.028 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:33.028 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.028 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:33.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:33.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:33.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:33.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_is_making_recovery_progress td/ceph-helpers 2026-02-13T06:44:33.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: test_get_is_making_recovery_progress: local dir=td/ceph-helpers 2026-02-13T06:44:33.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: test_get_is_making_recovery_progress: setup td/ceph-helpers 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:33.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:33.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:33.031 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:33.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:33.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:33.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:33.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:33.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:33.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:33.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:33.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:33.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:33.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:33.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:33.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:33.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:33.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:33.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:33.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:33.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:33.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:33.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:33.041 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:33.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:33.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:33.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:33.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1338: test_get_is_making_recovery_progress: run_mon td/ceph-helpers a 2026-02-13T06:44:33.042 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:33.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:33.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:33.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:33.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:33.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T06:44:33.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:33.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:33.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:33.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:33.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:33.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:44:33.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.090 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.091 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:33.091 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:33.091 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:33.137 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:33.137 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:33.137 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:33.138 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:33.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: test_get_is_making_recovery_progress: run_mgr td/ceph-helpers x 2026-02-13T06:44:33.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:33.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:33.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:33.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:33.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:33.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:33.284 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:33.284 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:33.284 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:33.284 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:33.284 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.284 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.285 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:33.285 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:33.285 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:33.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: test_get_is_making_recovery_progress: get_is_making_recovery_progress 2026-02-13T06:44:33.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T06:44:33.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T06:44:33.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T06:44:33.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T06:44:33.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T06:44:33.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T06:44:33.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1341: test_get_is_making_recovery_progress: teardown td/ceph-helpers 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:33.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:33.511 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:33.511 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:33.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:33.512 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:33.514 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:33.514 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:33.514 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:33.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:33.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:33.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:33.516 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:33.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:33.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:33.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:33.519 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:33.519 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.519 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:33.521 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:33.521 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_last_scrub_stamp td/ceph-helpers 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1524: test_get_last_scrub_stamp: local dir=td/ceph-helpers 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: test_get_last_scrub_stamp: setup td/ceph-helpers 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:33.522 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:33.523 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:33.523 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:33.523 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:33.523 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:33.524 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:33.524 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:33.524 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:33.525 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:33.525 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:33.525 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:33.525 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:33.525 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:33.526 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:33.526 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:33.526 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:33.527 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:33.527 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:33.529 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:33.529 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.529 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.529 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:33.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:33.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:33.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:33.531 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:33.531 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.531 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:33.532 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:33.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:33.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:33.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:33.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: test_get_last_scrub_stamp: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:33.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:33.533 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:33.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:33.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:33.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:33.534 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:33.554 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:33.554 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:33.554 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:33.554 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:33.554 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.554 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.554 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:33.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:33.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:33.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:33.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:33.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:33.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:33.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:33.585 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:33.585 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:33.585 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:33.585 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:33.585 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.585 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.585 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:33.585 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:33.585 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:33.628 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:33.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:33.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:33.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:33.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:33.629 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:33.629 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:33.629 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:33.630 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:33.630 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.630 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.630 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:33.630 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:33.631 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:33.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: test_get_last_scrub_stamp: run_mgr td/ceph-helpers x 2026-02-13T06:44:33.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:33.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:33.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:33.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:33.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:33.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:33.776 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:33.776 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:33.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:33.777 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:33.777 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.777 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:33.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:33.778 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:33.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: test_get_last_scrub_stamp: run_osd td/ceph-helpers 0 2026-02-13T06:44:33.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:33.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:33.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:33.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:33.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:33.799 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:33.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:33.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:33.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:33.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:33.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:33.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:33.801 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:33.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5f87407c-a6eb-4f5d-a46f-e1734e68bc33 2026-02-13T06:44:33.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5f87407c-a6eb-4f5d-a46f-e1734e68bc33' 2026-02-13T06:44:33.801 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 5f87407c-a6eb-4f5d-a46f-e1734e68bc33 2026-02-13T06:44:33.802 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:33.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBRyI5p7WAzMBAA9teMJ/UZT3CkJfeutsUkwQ== 2026-02-13T06:44:33.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBRyI5p7WAzMBAA9teMJ/UZT3CkJfeutsUkwQ=="}' 2026-02-13T06:44:33.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5f87407c-a6eb-4f5d-a46f-e1734e68bc33 -i td/ceph-helpers/0/new.json 2026-02-13T06:44:33.917 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:33.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:33.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBRyI5p7WAzMBAA9teMJ/UZT3CkJfeutsUkwQ== --osd-uuid 5f87407c-a6eb-4f5d-a46f-e1734e68bc33 2026-02-13T06:44:33.943 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:33.942+0000 7f8bfb210900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:33.945 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:33.943+0000 7f8bfb210900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:33.946 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:33.944+0000 7f8bfb210900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:33.946 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:33.945+0000 7f8bfb210900 -1 bdev(0x5599694b4c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:33.946 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:33.945+0000 7f8bfb210900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:34.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:34.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:34.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:34.360 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:34.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:34.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:34.469 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:34.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:34.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:34.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:34.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:34.486 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:34.485+0000 7ff06c55d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:34.487 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:34.486+0000 7ff06c55d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:34.488 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:34.487+0000 7ff06c55d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:34.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:34.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:34.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:34.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:34.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:34.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:34.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:34.572 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:34.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:34.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:34.672 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:34.744 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:34.742+0000 7ff06c55d900 -1 Falling back to public interface 2026-02-13T06:44:35.138 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:35.136+0000 7ff06c55d900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:35.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:35.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:35.674 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:35.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:35.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:35.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:35.775 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1869938653,v1:127.0.0.1:6801/1869938653] [v2:127.0.0.1:6802/1869938653,v1:127.0.0.1:6803/1869938653] exists,up 5f87407c-a6eb-4f5d-a46f-e1734e68bc33 2026-02-13T06:44:35.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:35.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:35.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:35.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1530: test_get_last_scrub_stamp: create_rbd_pool 2026-02-13T06:44:35.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:44:35.876 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:44:35.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:44:35.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:44:36.124 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:44:36.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:44:37.133 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:44:37.416 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1531: test_get_last_scrub_stamp: wait_for_clean 2026-02-13T06:44:37.417 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:44:37.417 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:44:37.417 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:44:37.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:44:37.418 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:44:37.418 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:44:37.418 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:44:37.418 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:44:37.418 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:44:37.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:44:37.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:44:37.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:44:37.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:44:37.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:44:37.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:44:37.725 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:44:37.725 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:44:37.725 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:44:37.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:44:37.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:44:37.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:44:37.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:44:37.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:44:37.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:37.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:44:37.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:44:37.792 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:37.792 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:44:37.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:44:37.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:44:37.793 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:44:37.794 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:44:38.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:44:38.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:44:38.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:44:38.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:44:38.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:44:38.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:44:38.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:44:38.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:44:38.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:44:38.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:44:38.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:44:38.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:44:38.566 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:44:38.566 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:44:38.567 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:44:38.567 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:44:38.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:44:38.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:44:38.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:44:38.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: get_last_scrub_stamp 1.0 2026-02-13T06:44:38.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:44:38.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:44:38.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:44:38.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:44:39.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: stamp=2026-02-13T06:44:36.059608+0000 2026-02-13T06:44:39.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1533: test_get_last_scrub_stamp: test -n 2026-02-13T06:44:36.059608+0000 2026-02-13T06:44:39.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1534: test_get_last_scrub_stamp: teardown td/ceph-helpers 2026-02-13T06:44:39.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:39.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:39.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:39.106 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:39.106 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:39.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:39.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:39.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:39.212 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:39.212 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:39.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:39.213 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:39.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:39.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:39.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:39.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:39.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:39.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:39.216 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:39.217 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:39.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:39.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:39.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:39.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:39.222 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:39.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:39.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:39.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:39.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:39.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_not_primary td/ceph-helpers 2026-02-13T06:44:39.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: test_get_not_primary: local dir=td/ceph-helpers 2026-02-13T06:44:39.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1231: test_get_not_primary: setup td/ceph-helpers 2026-02-13T06:44:39.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:39.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:39.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:39.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:39.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:39.225 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:39.225 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:39.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:39.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:39.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:39.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:39.226 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:39.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:39.226 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:39.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:39.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:39.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:39.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:39.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:39.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:39.229 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:39.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:39.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:39.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:39.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:39.232 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:39.232 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:39.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:39.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:39.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:39.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:39.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:39.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:39.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:39.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:39.235 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:39.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:39.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:39.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:39.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: test_get_not_primary: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-13T06:44:39.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:39.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:39.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:39.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:39.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:39.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-13T06:44:39.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:39.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:39.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:39.256 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:39.256 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:39.256 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:39.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:39.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-13T06:44:39.284 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:39.285 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:39.285 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:39.285 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:39.285 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:39.285 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:39.286 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:39.286 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:39.286 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:39.286 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:39.286 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:39.286 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:39.287 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:39.287 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:39.287 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:39.331 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:39.331 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:39.331 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:39.331 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:39.332 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:39.332 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:39.332 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:39.332 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:39.333 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:39.333 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:39.333 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:39.333 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:39.333 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:39.333 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:39.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: test_get_not_primary: run_mgr td/ceph-helpers x 2026-02-13T06:44:39.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:39.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:39.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:39.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:39.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:39.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:39.482 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:39.483 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:39.483 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:39.483 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:39.483 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:39.483 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:39.483 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:39.484 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:39.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:39.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: test_get_not_primary: run_osd td/ceph-helpers 0 2026-02-13T06:44:39.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:39.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:39.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:39.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:39.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:39.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:39.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:39.505 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:39.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:39.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:39.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:39.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:39.507 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:39.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=904abcda-5f36-4cf4-af6d-c22b8a9c742b 2026-02-13T06:44:39.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 904abcda-5f36-4cf4-af6d-c22b8a9c742b' 2026-02-13T06:44:39.508 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 904abcda-5f36-4cf4-af6d-c22b8a9c742b 2026-02-13T06:44:39.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:39.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBXyI5pU3GyHhAAj3pFvHaE9kV+ZOxTlbUayQ== 2026-02-13T06:44:39.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBXyI5pU3GyHhAAj3pFvHaE9kV+ZOxTlbUayQ=="}' 2026-02-13T06:44:39.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 904abcda-5f36-4cf4-af6d-c22b8a9c742b -i td/ceph-helpers/0/new.json 2026-02-13T06:44:39.622 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:39.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:39.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBXyI5pU3GyHhAAj3pFvHaE9kV+ZOxTlbUayQ== --osd-uuid 904abcda-5f36-4cf4-af6d-c22b8a9c742b 2026-02-13T06:44:39.648 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:39.646+0000 7f14aafc1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:39.649 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:39.648+0000 7f14aafc1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:39.650 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:39.649+0000 7f14aafc1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:39.650 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:39.649+0000 7f14aafc1900 -1 bdev(0x558c7ef4ec00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:39.650 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:39.649+0000 7f14aafc1900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:40.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:40.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:40.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:40.083 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:40.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:40.189 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:40.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:40.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:40.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:40.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:40.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:40.206 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:40.205+0000 7efe82697900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:40.209 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:40.208+0000 7efe82697900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:40.210 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:40.209+0000 7efe82697900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:40.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:40.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:40.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:40.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:40.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:40.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:40.295 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:40.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:40.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:40.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:40.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:40.417 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:40.415+0000 7efe82697900 -1 Falling back to public interface 2026-02-13T06:44:40.721 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:40.719+0000 7efe82697900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:41.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:41.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:41.397 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:41.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:41.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:41.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:41.500 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/557901999,v1:127.0.0.1:6801/557901999] [v2:127.0.0.1:6802/557901999,v1:127.0.0.1:6803/557901999] exists,up 904abcda-5f36-4cf4-af6d-c22b8a9c742b 2026-02-13T06:44:41.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:41.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:41.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:41.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1235: test_get_not_primary: run_osd td/ceph-helpers 1 2026-02-13T06:44:41.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:41.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:41.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:41.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8d979b14-1c5e-4d74-a322-71e646e4cddf 2026-02-13T06:44:41.507 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 8d979b14-1c5e-4d74-a322-71e646e4cddf 2026-02-13T06:44:41.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 8d979b14-1c5e-4d74-a322-71e646e4cddf' 2026-02-13T06:44:41.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:41.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZyI5pj4yvHhAAEwj3I3vlT86rl3bppOAz5A== 2026-02-13T06:44:41.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZyI5pj4yvHhAAEwj3I3vlT86rl3bppOAz5A=="}' 2026-02-13T06:44:41.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8d979b14-1c5e-4d74-a322-71e646e4cddf -i td/ceph-helpers/1/new.json 2026-02-13T06:44:41.623 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:41.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T06:44:41.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZyI5pj4yvHhAAEwj3I3vlT86rl3bppOAz5A== --osd-uuid 8d979b14-1c5e-4d74-a322-71e646e4cddf 2026-02-13T06:44:41.648 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:41.647+0000 7f7076e0d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:41.649 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:41.648+0000 7f7076e0d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:41.650 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:41.649+0000 7f7076e0d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:41.651 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:41.649+0000 7f7076e0d900 -1 bdev(0x55626744ec00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:41.651 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:41.650+0000 7f7076e0d900 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T06:44:41.983 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T06:44:41.983 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:41.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T06:44:41.984 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T06:44:41.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:42.304 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T06:44:42.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T06:44:42.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:42.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:42.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:42.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:42.321 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:42.320+0000 7f94bacd9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:42.322 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:42.321+0000 7f94bacd9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:42.323 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:42.322+0000 7f94bacd9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:42.448 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:42.446+0000 7f94bacd9900 -1 Falling back to public interface 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:42.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:44:42.756 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:42.753+0000 7f94bacd9900 -1 osd.1 0 log_to_monitors true 2026-02-13T06:44:42.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:43.762 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:43.762 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:43.762 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:43.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:43.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:43.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:44:43.990 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3956137435,v1:127.0.0.1:6811/3956137435] [v2:127.0.0.1:6812/3956137435,v1:127.0.0.1:6813/3956137435] exists,up 8d979b14-1c5e-4d74-a322-71e646e4cddf 2026-02-13T06:44:43.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:43.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:43.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:43.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1236: test_get_not_primary: create_rbd_pool 2026-02-13T06:44:43.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:44:44.217 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:44:44.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:44:44.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:44:44.497 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:44:44.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:44:45.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:44:45.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1237: test_get_not_primary: wait_for_clean 2026-02-13T06:44:45.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:44:45.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:44:45.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:44:45.789 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:44:45.789 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:44:45.789 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:44:45.789 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:44:45.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:44:45.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:44:45.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:44:45.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:44:45.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:44:45.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:44:45.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:44:45.871 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:44:46.096 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:44:46.096 INFO:tasks.workunit.client.0.trial040.stderr:1' 2026-02-13T06:44:46.097 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:44:46.097 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:44:46.097 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:44:46.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:44:46.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:44:46.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:44:46.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:44:46.164 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:44:46.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-13T06:44:46.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-13T06:44:46.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667' 2026-02-13T06:44:46.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:44:46.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:46.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:44:46.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:44:46.232 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:46.232 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:44:46.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:44:46.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:44:46.233 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:44:46.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:44:46.463 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:44:46.463 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:44:47.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:44:47.465 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:44:47.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-13T06:44:47.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:44:47.695 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-13T06:44:47.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:44:47.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:44:47.697 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-13T06:44:47.697 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:44:47.698 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-13T06:44:47.698 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 38654705667 2026-02-13T06:44:47.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-13T06:44:47.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:44:47.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-13T06:44:47.928 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:44:47.928 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:44:47.928 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:44:48.240 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:44:48.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:44:48.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:44:48.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:44:48.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:44:48.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:44:48.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:44:48.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:44:48.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:44:48.465 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:44:48.466 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:44:48.466 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:44:48.777 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:44:48.777 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:44:48.777 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:44:48.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: get_primary rbd GROUP 2026-02-13T06:44:48.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-13T06:44:48.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-13T06:44:48.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-13T06:44:48.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-13T06:44:49.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: local primary=1 2026-02-13T06:44:49.008 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: get_not_primary rbd GROUP 2026-02-13T06:44:49.008 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-13T06:44:49.008 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=GROUP 2026-02-13T06:44:49.008 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd GROUP 2026-02-13T06:44:49.008 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-13T06:44:49.008 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-13T06:44:49.008 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-13T06:44:49.009 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-13T06:44:49.240 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-13T06:44:49.240 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd GROUP 2026-02-13T06:44:49.240 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: local not_primary=0 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1240: test_get_not_primary: test 0 '!=' 1 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1241: test_get_not_primary: test 0 = 0 -o 0 = 1 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1242: test_get_not_primary: teardown td/ceph-helpers 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:49.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:49.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:49.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:49.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:49.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:49.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:49.584 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:49.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:49.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:49.586 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:49.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:49.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:49.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:49.588 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:49.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:49.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:49.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:49.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:49.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:49.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:49.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:49.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:49.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_num_active_clean td/ceph-helpers 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: test_get_num_active_clean: local dir=td/ceph-helpers 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: test_get_num_active_clean: setup td/ceph-helpers 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:49.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:49.598 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:49.598 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:49.598 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:49.598 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:49.599 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:49.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:49.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:49.600 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:49.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:49.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:49.601 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:49.601 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:49.601 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:49.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:49.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:49.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:49.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:49.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:49.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:49.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:49.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:49.605 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:49.606 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:49.606 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:49.606 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:49.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:49.607 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:49.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:49.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:49.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:49.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: test_get_num_active_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:49.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:49.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:49.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:49.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:49.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:49.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:49.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:49.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:49.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:49.629 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:49.629 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:49.629 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:49.630 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:49.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:49.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:49.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:49.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:49.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:49.659 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:49.659 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:49.659 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:49.659 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:49.659 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:49.660 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:49.660 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:49.660 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:49.660 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:49.660 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:49.660 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:49.703 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:49.703 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:49.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:49.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:49.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:49.705 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:49.705 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:49.705 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:49.705 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:49.705 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:49.705 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:49.705 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:49.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:49.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:49.751 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: test_get_num_active_clean: run_mgr td/ceph-helpers x 2026-02-13T06:44:49.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:49.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:49.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:49.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:49.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:49.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:49.855 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:49.855 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:49.856 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:49.856 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:49.856 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:49.856 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:49.856 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:49.856 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:49.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:49.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: test_get_num_active_clean: run_osd td/ceph-helpers 0 2026-02-13T06:44:49.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:49.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:49.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:49.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:49.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:49.878 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:49.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:49.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:49.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:49.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:49.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:49.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:49.880 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:49.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=652a5c27-17b7-4ee1-be6b-fd21732a5252 2026-02-13T06:44:49.880 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 652a5c27-17b7-4ee1-be6b-fd21732a5252 2026-02-13T06:44:49.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 652a5c27-17b7-4ee1-be6b-fd21732a5252' 2026-02-13T06:44:49.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:49.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBhyI5pV6/rNBAAJ/TEpfkyRKq6kFEVxzUmhg== 2026-02-13T06:44:49.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBhyI5pV6/rNBAAJ/TEpfkyRKq6kFEVxzUmhg=="}' 2026-02-13T06:44:49.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 652a5c27-17b7-4ee1-be6b-fd21732a5252 -i td/ceph-helpers/0/new.json 2026-02-13T06:44:49.993 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:49.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:50.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBhyI5pV6/rNBAAJ/TEpfkyRKq6kFEVxzUmhg== --osd-uuid 652a5c27-17b7-4ee1-be6b-fd21732a5252 2026-02-13T06:44:50.018 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:50.016+0000 7f1ff1627900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:50.019 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:50.017+0000 7f1ff1627900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:50.020 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:50.018+0000 7f1ff1627900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:50.021 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:50.018+0000 7f1ff1627900 -1 bdev(0x55b4ef60ec00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:50.021 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:50.018+0000 7f1ff1627900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:50.452 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:50.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:50.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:50.454 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:50.454 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:50.563 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:50.563 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:50.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:50.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:50.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:50.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:50.580 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:50.578+0000 7f03a7ea1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:50.581 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:50.579+0000 7f03a7ea1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:50.582 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:50.580+0000 7f03a7ea1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:50.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:50.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:50.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:50.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:50.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:50.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:50.664 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:50.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:50.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:50.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:50.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:50.782 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:50.780+0000 7f03a7ea1900 -1 Falling back to public interface 2026-02-13T06:44:51.129 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:51.127+0000 7f03a7ea1900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:51.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:51.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:51.766 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:51.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:51.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:51.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:51.867 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3034961176,v1:127.0.0.1:6801/3034961176] [v2:127.0.0.1:6802/3034961176,v1:127.0.0.1:6803/3034961176] exists,up 652a5c27-17b7-4ee1-be6b-fd21732a5252 2026-02-13T06:44:51.868 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:51.868 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:51.868 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:51.868 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1369: test_get_num_active_clean: create_rbd_pool 2026-02-13T06:44:51.868 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:44:51.967 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:44:51.973 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:44:51.973 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:44:52.211 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:44:52.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:44:53.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:44:53.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1370: test_get_num_active_clean: wait_for_clean 2026-02-13T06:44:53.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:44:53.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:44:53.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:44:53.505 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:44:53.505 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:44:53.505 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:44:53.505 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:44:53.505 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:44:53.505 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:44:53.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:44:53.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:44:53.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:44:53.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:44:53.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:44:53.585 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:44:53.811 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:44:53.812 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:44:53.812 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:44:53.812 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:44:53.878 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:44:53.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:44:53.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:44:53.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:44:53.879 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:53.879 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:44:53.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:44:53.880 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:53.880 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:44:53.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:44:53.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:44:53.882 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:44:53.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:44:54.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:44:54.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:44:54.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:44:54.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:44:54.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:44:54.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:44:54.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:44:54.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:44:54.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:44:54.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:44:54.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:44:54.420 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:44:54.640 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:44:54.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:44:54.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:44:54.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:44:54.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:44:54.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:44:54.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:44:54.955 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: get_num_active_clean 2026-02-13T06:44:54.955 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:44:54.955 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:44:54.955 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:44:54.955 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:44:54.955 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:44:55.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: local num_active_clean=4 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1372: test_get_num_active_clean: test 4 = 4 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1373: test_get_num_active_clean: teardown td/ceph-helpers 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:55.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:55.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:55.288 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:55.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:55.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:55.291 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:55.291 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:55.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:55.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:55.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:55.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:55.293 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:55.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:55.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:55.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:55.298 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:55.299 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:55.299 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:55.299 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:55.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:55.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:55.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:44:55.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_num_active_or_peered td/ceph-helpers 2026-02-13T06:44:55.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1393: test_get_num_active_or_peered: local dir=td/ceph-helpers 2026-02-13T06:44:55.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1395: test_get_num_active_or_peered: setup td/ceph-helpers 2026-02-13T06:44:55.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:44:55.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:44:55.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:44:55.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:44:55.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:44:55.302 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:44:55.302 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:44:55.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:44:55.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:44:55.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:44:55.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:44:55.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:44:55.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:44:55.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:44:55.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:44:55.304 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:44:55.304 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:44:55.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:55.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:44:55.305 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:44:55.306 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:44:55.306 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:44:55.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:44:55.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:44:55.308 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:44:55.308 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:55.308 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:55.308 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:44:55.309 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:44:55.309 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:44:55.309 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:44:55.310 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:44:55.310 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:55.310 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:55.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:44:55.311 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:44:55.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:44:55.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:44:55.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:44:55.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1396: test_get_num_active_or_peered: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:55.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:44:55.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:44:55.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:44:55.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:44:55.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:44:55.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:55.334 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:44:55.334 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:55.334 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:55.334 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:55.334 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:55.334 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:55.334 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:55.335 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:44:55.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:44:55.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:44:55.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:55.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:55.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:44:55.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:44:55.363 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:55.364 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:55.364 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:55.364 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:55.364 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:55.364 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:55.364 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:55.365 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:55.365 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:44:55.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:44:55.409 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:44:55.409 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:44:55.409 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:44:55.410 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:44:55.410 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:44:55.410 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:44:55.410 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:44:55.410 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:44:55.411 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:55.411 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:55.411 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:44:55.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:44:55.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:44:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1397: test_get_num_active_or_peered: run_mgr td/ceph-helpers x 2026-02-13T06:44:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:44:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:44:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:44:55.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:44:55.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:44:55.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:44:55.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:44:55.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:55.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:55.559 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:55.560 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:55.560 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:55.560 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:55.560 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:55.561 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:44:55.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1398: test_get_num_active_or_peered: run_osd td/ceph-helpers 0 2026-02-13T06:44:55.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:44:55.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:44:55.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:44:55.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:44:55.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:44:55.585 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:44:55.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a16d8659-d132-460f-b5fa-d5a51fafabd8 2026-02-13T06:44:55.585 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 a16d8659-d132-460f-b5fa-d5a51fafabd8 2026-02-13T06:44:55.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a16d8659-d132-460f-b5fa-d5a51fafabd8' 2026-02-13T06:44:55.586 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:44:55.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBnyI5pfw9SIxAARpLh4T8zvAhHJXP0VaHxOw== 2026-02-13T06:44:55.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBnyI5pfw9SIxAARpLh4T8zvAhHJXP0VaHxOw=="}' 2026-02-13T06:44:55.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a16d8659-d132-460f-b5fa-d5a51fafabd8 -i td/ceph-helpers/0/new.json 2026-02-13T06:44:55.701 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:55.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:44:55.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBnyI5pfw9SIxAARpLh4T8zvAhHJXP0VaHxOw== --osd-uuid a16d8659-d132-460f-b5fa-d5a51fafabd8 2026-02-13T06:44:55.727 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:55.724+0000 7f137780f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:55.728 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:55.726+0000 7f137780f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:55.729 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:55.727+0000 7f137780f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:55.729 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:55.727+0000 7f137780f900 -1 bdev(0x55631d700c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:44:55.730 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:55.727+0000 7f137780f900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:44:56.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:44:56.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:44:56.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:44:56.160 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:44:56.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:44:56.263 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:44:56.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:44:56.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:44:56.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:44:56.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:44:56.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:44:56.280 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:56.278+0000 7f61d6164900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:56.282 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:56.279+0000 7f61d6164900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:56.283 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:56.281+0000 7f61d6164900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:44:56.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:44:56.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:44:56.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:44:56.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:44:56.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:44:56.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:56.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:44:56.366 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:44:56.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:56.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:56.467 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:44:56.486 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:56.484+0000 7f61d6164900 -1 Falling back to public interface 2026-02-13T06:44:56.787 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:44:56.785+0000 7f61d6164900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:44:57.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:44:57.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:44:57.469 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:44:57.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:44:57.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:44:57.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:44:57.570 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/889639248,v1:127.0.0.1:6801/889639248] [v2:127.0.0.1:6802/889639248,v1:127.0.0.1:6803/889639248] exists,up a16d8659-d132-460f-b5fa-d5a51fafabd8 2026-02-13T06:44:57.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:44:57.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:44:57.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:44:57.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1399: test_get_num_active_or_peered: create_rbd_pool 2026-02-13T06:44:57.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:44:57.669 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:44:57.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:44:57.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:44:57.928 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:44:57.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:44:58.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:44:59.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1400: test_get_num_active_or_peered: wait_for_clean 2026-02-13T06:44:59.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:44:59.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:44:59.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:44:59.218 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:44:59.219 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:44:59.219 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:44:59.219 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:44:59.219 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:44:59.219 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:44:59.298 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:44:59.298 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:44:59.298 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:44:59.298 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:44:59.298 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:44:59.298 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:44:59.532 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:44:59.532 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:44:59.532 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:44:59.533 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:44:59.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:44:59.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:44:59.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:44:59.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:44:59.600 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:59.600 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:44:59.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:44:59.601 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:44:59.601 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:44:59.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:44:59.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:44:59.602 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:44:59.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:44:59.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-13T06:44:59.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:00.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:45:00.836 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:01.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-13T06:45:01.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:02.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T06:45:02.067 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:02.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:45:02.296 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:45:02.296 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:02.296 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:02.613 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:45:02.613 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:45:02.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:45:02.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:45:02.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:45:02.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:45:02.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:45:02.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:45:02.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:45:02.838 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:45:02.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:02.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:03.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:45:03.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:45:03.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:45:03.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: get_num_active_or_peered 2026-02-13T06:45:03.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-13T06:45:03.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-13T06:45:03.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-13T06:45:03.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-13T06:45:03.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-13T06:45:03.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: local num_peered=4 2026-02-13T06:45:03.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1402: test_get_num_active_or_peered: test 4 = 4 2026-02-13T06:45:03.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1403: test_get_num_active_or_peered: teardown td/ceph-helpers 2026-02-13T06:45:03.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:03.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:03.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:03.376 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:03.376 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:03.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:03.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:03.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:03.482 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:03.483 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:03.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:03.484 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:03.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:03.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:03.486 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:03.487 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:03.487 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:03.487 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:03.487 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:03.488 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:03.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:03.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:03.492 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:03.492 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:03.492 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:03.493 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_num_pgs td/ceph-helpers 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1420: test_get_num_pgs: local dir=td/ceph-helpers 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1422: test_get_num_pgs: setup td/ceph-helpers 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:03.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:03.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:03.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:03.497 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:03.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:03.497 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:03.497 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:03.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:03.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:03.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:03.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:03.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:03.499 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:03.499 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:03.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:03.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:03.502 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:03.502 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:03.502 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:03.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:03.503 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:03.503 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:03.503 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:45:03.504 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:45:03.504 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:03.504 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:03.504 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:45:03.505 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:45:03.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:45:03.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:45:03.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:45:03.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1423: test_get_num_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:03.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:45:03.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:45:03.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:45:03.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:45:03.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:45:03.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:03.528 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:45:03.528 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:03.528 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:03.528 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:03.528 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:03.528 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:03.528 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:03.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:03.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:45:03.556 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:03.557 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:03.558 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:03.558 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:45:03.601 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:45:03.601 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:03.602 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:03.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:03.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:45:03.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1424: test_get_num_pgs: run_mgr td/ceph-helpers x 2026-02-13T06:45:03.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:45:03.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:45:03.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:45:03.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:45:03.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:45:03.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:45:03.751 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:45:03.751 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:03.751 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:03.751 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:03.751 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:03.751 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:03.751 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:03.752 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:03.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:03.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: test_get_num_pgs: run_osd td/ceph-helpers 0 2026-02-13T06:45:03.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:45:03.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:45:03.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:45:03.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:45:03.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:45:03.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:45:03.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:03.774 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:45:03.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:45:03.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:45:03.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:45:03.776 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:45:03.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=35f0fea3-fb61-4c5c-88a0-4dec7f104d96 2026-02-13T06:45:03.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 35f0fea3-fb61-4c5c-88a0-4dec7f104d96' 2026-02-13T06:45:03.776 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 35f0fea3-fb61-4c5c-88a0-4dec7f104d96 2026-02-13T06:45:03.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:45:03.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBvyI5pbGGmLhAAAg5xBhTbHVFLE5H+JESiSw== 2026-02-13T06:45:03.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBvyI5pbGGmLhAAAg5xBhTbHVFLE5H+JESiSw=="}' 2026-02-13T06:45:03.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 35f0fea3-fb61-4c5c-88a0-4dec7f104d96 -i td/ceph-helpers/0/new.json 2026-02-13T06:45:03.889 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:03.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:45:03.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBvyI5pbGGmLhAAAg5xBhTbHVFLE5H+JESiSw== --osd-uuid 35f0fea3-fb61-4c5c-88a0-4dec7f104d96 2026-02-13T06:45:03.914 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:03.912+0000 7f7b9b78a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:03.916 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:03.913+0000 7f7b9b78a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:03.916 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:03.914+0000 7f7b9b78a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:03.917 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:03.914+0000 7f7b9b78a900 -1 bdev(0x55f5d56c0c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:45:03.917 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:03.915+0000 7f7b9b78a900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:45:04.336 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:45:04.336 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:45:04.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:45:04.337 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:45:04.338 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:45:04.445 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:45:04.445 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:45:04.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:45:04.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:45:04.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:45:04.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:45:04.462 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:04.460+0000 7f675fe5d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:04.464 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:04.461+0000 7f675fe5d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:04.465 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:04.462+0000 7f675fe5d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:04.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:45:04.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:45:04.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:45:04.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:45:04.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:45:04.549 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:04.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:04.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:45:04.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:04.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:04.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:45:04.672 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:04.670+0000 7f675fe5d900 -1 Falling back to public interface 2026-02-13T06:45:05.089 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:05.086+0000 7f675fe5d900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:45:05.649 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:45:05.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:05.650 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:05.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:45:05.651 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:05.651 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:05.757 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/425036819,v1:127.0.0.1:6801/425036819] [v2:127.0.0.1:6802/425036819,v1:127.0.0.1:6803/425036819] exists,up 35f0fea3-fb61-4c5c-88a0-4dec7f104d96 2026-02-13T06:45:05.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:45:05.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:45:05.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:45:05.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1426: test_get_num_pgs: create_rbd_pool 2026-02-13T06:45:05.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:45:05.856 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:45:05.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:45:05.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:45:06.140 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:45:06.148 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:45:07.149 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:45:07.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1427: test_get_num_pgs: wait_for_clean 2026-02-13T06:45:07.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:45:07.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:45:07.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:45:07.431 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:45:07.432 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:07.432 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:07.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:07.433 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:07.433 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:07.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:45:07.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:45:07.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:45:07.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:45:07.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:45:07.513 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:45:07.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:45:07.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:45:07.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:45:07.749 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:45:07.813 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:45:07.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:45:07.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:45:07.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:45:07.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:07.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:45:07.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:45:07.815 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:07.815 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:45:07.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:45:07.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:45:07.817 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:45:07.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:08.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:45:08.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:09.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:45:09.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:09.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:45:09.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:10.274 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T06:45:10.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:10.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:45:10.503 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:45:10.503 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:10.503 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:10.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:45:10.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:45:10.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:45:10.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:45:10.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:45:10.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:45:10.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:45:10.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:45:11.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:45:11.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:45:11.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:11.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:11.373 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:45:11.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:45:11.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:45:11.374 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: get_num_pgs 2026-02-13T06:45:11.374 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:11.375 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:11.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: local num_pgs=4 2026-02-13T06:45:11.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1429: test_get_num_pgs: test 4 -gt 0 2026-02-13T06:45:11.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1430: test_get_num_pgs: teardown td/ceph-helpers 2026-02-13T06:45:11.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:11.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:11.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:11.688 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:11.688 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:11.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:11.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:11.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:11.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:11.794 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:11.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:11.795 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:11.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:11.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:11.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:11.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:11.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:11.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:11.799 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:11.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:11.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:11.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:11.803 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:11.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:11.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:11.804 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:11.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:11.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:11.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:45:11.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_op_scheduler td/ceph-helpers 2026-02-13T06:45:11.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2316: test_get_op_scheduler: local dir=td/ceph-helpers 2026-02-13T06:45:11.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2318: test_get_op_scheduler: setup td/ceph-helpers 2026-02-13T06:45:11.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:45:11.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:45:11.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:11.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:11.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:11.807 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:11.807 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:11.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:11.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:11.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:11.808 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:11.808 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:11.808 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:11.808 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:11.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:11.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:11.809 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:11.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:11.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:11.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:11.811 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:11.811 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:11.812 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:11.812 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:11.813 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:11.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:11.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:11.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:11.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:11.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:11.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:45:11.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:45:11.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:11.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:11.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:45:11.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:45:11.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:45:11.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:45:11.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:45:11.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2320: test_get_op_scheduler: run_mon td/ceph-helpers a 2026-02-13T06:45:11.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:45:11.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:45:11.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:45:11.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:45:11.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:45:11.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T06:45:11.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:45:11.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:11.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:11.839 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:11.839 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:11.839 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:11.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:11.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T06:45:11.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:45:11.866 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:11.867 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:11.868 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:11.868 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:45:11.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:45:11.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:11.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:11.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:45:11.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:45:11.912 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:11.912 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:11.912 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:11.913 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:11.914 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:11.914 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:11.914 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:11.914 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:11.914 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:45:11.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2321: test_get_op_scheduler: run_mgr td/ceph-helpers x 2026-02-13T06:45:11.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:45:11.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:45:11.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:45:11.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:45:11.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:45:11.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:45:12.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:45:12.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:12.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:12.059 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:12.059 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:12.060 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:12.060 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:12.060 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:12.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:12.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: run_osd td/ceph-helpers 0 --osd_op_queue=wpq 2026-02-13T06:45:12.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:45:12.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:45:12.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:45:12.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:45:12.080 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:45:12.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:45:12.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:45:12.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:45:12.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:45:12.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=wpq 2026-02-13T06:45:12.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:45:12.082 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:45:12.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0352da9c-74a5-43c0-a936-3b2acddfd25e 2026-02-13T06:45:12.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0352da9c-74a5-43c0-a936-3b2acddfd25e' 2026-02-13T06:45:12.083 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 0352da9c-74a5-43c0-a936-3b2acddfd25e 2026-02-13T06:45:12.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:45:12.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB4yI5pXmdfBRAAaYHoO5fXBZHJ5qjvgF5wSA== 2026-02-13T06:45:12.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB4yI5pXmdfBRAAaYHoO5fXBZHJ5qjvgF5wSA=="}' 2026-02-13T06:45:12.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0352da9c-74a5-43c0-a936-3b2acddfd25e -i td/ceph-helpers/0/new.json 2026-02-13T06:45:12.194 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:12.201 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:45:12.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq --mkfs --key AQB4yI5pXmdfBRAAaYHoO5fXBZHJ5qjvgF5wSA== --osd-uuid 0352da9c-74a5-43c0-a936-3b2acddfd25e 2026-02-13T06:45:12.221 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:12.220+0000 7f5d1e1c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:12.222 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:12.221+0000 7f5d1e1c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:12.223 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:12.222+0000 7f5d1e1c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:12.224 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:12.222+0000 7f5d1e1c6900 -1 bdev(0x5639f2db0c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:45:12.224 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:12.222+0000 7f5d1e1c6900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:45:12.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:45:12.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:45:12.644 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:45:12.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:45:12.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:45:12.751 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:45:12.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:45:12.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq 2026-02-13T06:45:12.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:45:12.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:45:12.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:45:12.768 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:12.767+0000 7fb59076b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:12.770 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:12.768+0000 7fb59076b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:12.770 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:12.769+0000 7fb59076b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:12.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:45:12.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:45:12.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:45:12.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:45:12.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:45:12.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:12.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:45:12.852 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:12.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:12.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:12.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:45:12.997 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:12.996+0000 7fb59076b900 -1 Falling back to public interface 2026-02-13T06:45:13.364 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:13.362+0000 7fb59076b900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:45:13.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:45:13.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:13.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:45:13.954 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:13.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:13.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:14.055 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2988355184,v1:127.0.0.1:6801/2988355184] [v2:127.0.0.1:6802/2988355184,v1:127.0.0.1:6803/2988355184] exists,up 0352da9c-74a5-43c0-a936-3b2acddfd25e 2026-02-13T06:45:14.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:45:14.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:45:14.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:45:14.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2324: test_get_op_scheduler: get_op_scheduler 0 2026-02-13T06:45:14.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2310: get_op_scheduler: local id=0 2026-02-13T06:45:14.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2312: get_op_scheduler: get_config osd 0 osd_op_queue 2026-02-13T06:45:14.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:14.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_op_queue 2026-02-13T06:45:14.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2324: test_get_op_scheduler: test wpq = wpq 2026-02-13T06:45:14.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: run_osd td/ceph-helpers 1 --osd_op_queue=mclock_scheduler 2026-02-13T06:45:14.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:45:14.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:45:14.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T06:45:14.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:45:14.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T06:45:14.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:45:14.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:14.112 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:45:14.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:45:14.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T06:45:14.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T06:45:14.114 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:45:14.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d1e95281-c0c8-4fd2-85b8-a424ab475c72 2026-02-13T06:45:14.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d1e95281-c0c8-4fd2-85b8-a424ab475c72' 2026-02-13T06:45:14.114 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 d1e95281-c0c8-4fd2-85b8-a424ab475c72 2026-02-13T06:45:14.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:45:14.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB6yI5pUG5QBxAAHvLC5G6zLrBWwTIix2zgzg== 2026-02-13T06:45:14.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB6yI5pUG5QBxAAHvLC5G6zLrBWwTIix2zgzg=="}' 2026-02-13T06:45:14.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d1e95281-c0c8-4fd2-85b8-a424ab475c72 -i td/ceph-helpers/1/new.json 2026-02-13T06:45:14.231 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:14.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T06:45:14.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQB6yI5pUG5QBxAAHvLC5G6zLrBWwTIix2zgzg== --osd-uuid d1e95281-c0c8-4fd2-85b8-a424ab475c72 2026-02-13T06:45:14.256 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:14.254+0000 7f70d1583900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:14.257 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:14.256+0000 7f70d1583900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:14.258 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:14.257+0000 7f70d1583900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:14.258 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:14.257+0000 7f70d1583900 -1 bdev(0x55d6849b6c00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T06:45:14.258 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:14.257+0000 7f70d1583900 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T06:45:14.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T06:45:14.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:45:14.576 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T06:45:14.576 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T06:45:14.577 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:45:14.896 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T06:45:14.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T06:45:14.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T06:45:14.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:45:14.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:45:14.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:45:14.915 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:14.913+0000 7fe27df64900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:14.916 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:14.915+0000 7fe27df64900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:14.917 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:14.916+0000 7fe27df64900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:15.046 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:15.045+0000 7fe27df64900 -1 Falling back to public interface 2026-02-13T06:45:15.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T06:45:15.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:45:15.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T06:45:15.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:45:15.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:45:15.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:15.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:45:15.124 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:15.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:15.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:45:15.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:45:15.409 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:15.408+0000 7fe27df64900 -1 osd.1 0 log_to_monitors true 2026-02-13T06:45:16.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:45:16.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:16.350 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:16.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:45:16.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:16.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:45:16.582 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/808254338,v1:127.0.0.1:6811/808254338] [v2:127.0.0.1:6812/808254338,v1:127.0.0.1:6813/808254338] exists,up d1e95281-c0c8-4fd2-85b8-a424ab475c72 2026-02-13T06:45:16.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:45:16.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2327: test_get_op_scheduler: get_op_scheduler 1 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2310: get_op_scheduler: local id=1 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2312: get_op_scheduler: get_config osd 1 osd_op_queue 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=1 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.1 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T06:45:16.596 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:16.597 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:16.597 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:16.597 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.1.asok 2026-02-13T06:45:16.597 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:16.597 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-osd.1.asok config get osd_op_queue 2026-02-13T06:45:16.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2327: test_get_op_scheduler: test mclock_scheduler = mclock_scheduler 2026-02-13T06:45:16.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2328: test_get_op_scheduler: teardown td/ceph-helpers 2026-02-13T06:45:16.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:16.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:16.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:16.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:16.635 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:16.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:16.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:16.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:16.741 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:16.741 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:16.742 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:16.742 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:16.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:16.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:16.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:16.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:16.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:16.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:16.746 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:16.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:16.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:16.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:16.752 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:16.752 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:16.752 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:16.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:16.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:16.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:16.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:45:16.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_osd_id_used_by_pgs td/ceph-helpers 2026-02-13T06:45:16.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1450: test_get_osd_id_used_by_pgs: local dir=td/ceph-helpers 2026-02-13T06:45:16.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1452: test_get_osd_id_used_by_pgs: setup td/ceph-helpers 2026-02-13T06:45:16.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:45:16.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:45:16.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:16.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:16.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:16.755 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:16.755 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:16.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:16.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:16.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:16.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:16.756 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:16.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:16.756 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:16.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:16.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:16.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:16.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:16.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:16.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:16.759 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:16.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:16.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:16.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:16.761 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:16.761 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:16.762 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:16.762 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:16.762 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:16.762 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:16.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:45:16.763 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:45:16.764 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:16.764 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:16.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:45:16.764 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:45:16.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:45:16.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:45:16.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:45:16.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1453: test_get_osd_id_used_by_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:16.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:45:16.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:45:16.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:45:16.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:45:16.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:45:16.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:16.787 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:45:16.787 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:16.787 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:16.787 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:16.787 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:16.788 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:16.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:16.788 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:16.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:45:16.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:45:16.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:16.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:16.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:45:16.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:45:16.817 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:16.818 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:16.818 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:16.818 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:16.818 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:16.818 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:16.818 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:16.818 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:16.818 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:45:16.860 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:45:16.861 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:16.861 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:16.861 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:45:16.861 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:45:16.861 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:16.861 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:16.861 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:16.862 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:16.862 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:16.862 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:16.862 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:16.862 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:16.863 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:45:16.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1454: test_get_osd_id_used_by_pgs: run_mgr td/ceph-helpers x 2026-02-13T06:45:16.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:45:16.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:45:16.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:45:16.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:45:16.907 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:45:16.907 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:45:17.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:45:17.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:17.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:17.010 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:17.010 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:17.010 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:17.010 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:17.010 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:17.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:17.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1455: test_get_osd_id_used_by_pgs: run_osd td/ceph-helpers 0 2026-02-13T06:45:17.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:45:17.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:45:17.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:45:17.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:45:17.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:17.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:45:17.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:45:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:45:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:45:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:45:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:45:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:45:17.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:45:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2693ac14-ba00-4154-ab55-da3546bfda92 2026-02-13T06:45:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2693ac14-ba00-4154-ab55-da3546bfda92' 2026-02-13T06:45:17.034 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 2693ac14-ba00-4154-ab55-da3546bfda92 2026-02-13T06:45:17.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:45:17.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB9yI5pB2p8AhAAAxYEB/RAiec45kMUFMju7A== 2026-02-13T06:45:17.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB9yI5pB2p8AhAAAxYEB/RAiec45kMUFMju7A=="}' 2026-02-13T06:45:17.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2693ac14-ba00-4154-ab55-da3546bfda92 -i td/ceph-helpers/0/new.json 2026-02-13T06:45:17.149 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:17.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:45:17.156 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB9yI5pB2p8AhAAAxYEB/RAiec45kMUFMju7A== --osd-uuid 2693ac14-ba00-4154-ab55-da3546bfda92 2026-02-13T06:45:17.175 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:17.174+0000 7fa21c07f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:17.176 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:17.175+0000 7fa21c07f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:17.177 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:17.176+0000 7fa21c07f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:17.178 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:17.176+0000 7fa21c07f900 -1 bdev(0x5618c0f14c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:45:17.178 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:17.177+0000 7fa21c07f900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:45:17.652 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:45:17.652 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:45:17.652 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:45:17.653 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:45:17.653 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:45:17.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:45:17.757 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:45:17.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:45:17.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:45:17.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:45:17.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:45:17.774 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:17.773+0000 7efe56622900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:17.776 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:17.774+0000 7efe56622900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:17.776 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:17.775+0000 7efe56622900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:17.856 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:45:17.856 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:45:17.856 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:45:17.856 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:45:17.856 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:45:17.856 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:17.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:45:17.857 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:17.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:17.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:17.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:45:17.969 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:17.968+0000 7efe56622900 -1 Falling back to public interface 2026-02-13T06:45:18.326 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:18.325+0000 7efe56622900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:45:18.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:45:18.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:18.959 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:18.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:45:18.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:18.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:19.062 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/757196949,v1:127.0.0.1:6801/757196949] [v2:127.0.0.1:6802/757196949,v1:127.0.0.1:6803/757196949] exists,up 2693ac14-ba00-4154-ab55-da3546bfda92 2026-02-13T06:45:19.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:45:19.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:45:19.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:45:19.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1456: test_get_osd_id_used_by_pgs: create_rbd_pool 2026-02-13T06:45:19.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:45:19.159 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:45:19.167 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:45:19.167 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:45:19.416 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:45:19.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:45:20.425 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:45:20.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1457: test_get_osd_id_used_by_pgs: wait_for_clean 2026-02-13T06:45:20.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:45:20.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:45:20.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:45:20.708 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:45:20.709 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:20.709 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:20.709 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:20.709 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:20.709 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:20.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:45:20.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:45:20.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:45:20.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:45:20.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:45:20.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:45:21.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:45:21.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:45:21.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:45:21.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:45:21.082 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:45:21.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:45:21.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:45:21.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:45:21.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:21.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:45:21.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:45:21.084 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:21.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:45:21.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:45:21.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:45:21.086 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:45:21.086 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:21.316 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:45:21.316 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:22.317 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:45:22.318 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:22.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:45:22.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:23.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T06:45:23.546 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:23.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:45:23.773 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:45:23.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:23.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:24.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:45:24.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:45:24.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:45:24.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:45:24.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:45:24.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:45:24.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:45:24.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:45:24.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:45:24.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:45:24.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:24.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:24.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:45:24.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:45:24.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:45:24.632 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: get_osd_id_used_by_pgs 2026-02-13T06:45:24.632 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: uniq 2026-02-13T06:45:24.632 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: ceph --format json pg dump pgs 2026-02-13T06:45:24.632 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: jq '.pg_stats | .[] | .up[], .acting[]' 2026-02-13T06:45:24.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: sort 2026-02-13T06:45:24.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: local osd_ids=0 2026-02-13T06:45:24.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1459: test_get_osd_id_used_by_pgs: test 0 = 0 2026-02-13T06:45:24.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1460: test_get_osd_id_used_by_pgs: teardown td/ceph-helpers 2026-02-13T06:45:24.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:24.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:24.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:24.857 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:24.858 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:24.858 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:24.858 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:24.858 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:24.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:24.964 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:24.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:24.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:24.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:24.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:24.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:24.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:24.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:24.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:24.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:24.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:24.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:24.974 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:24.974 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:24.974 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:24.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:24.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:24.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:24.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:45:24.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_osds td/ceph-helpers 2026-02-13T06:45:24.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: test_get_osds: local dir=td/ceph-helpers 2026-02-13T06:45:24.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1024: test_get_osds: setup td/ceph-helpers 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:24.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:24.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:24.978 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:24.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:24.979 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:24.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:24.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:24.980 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:24.980 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:24.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:24.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:24.981 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:24.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:24.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:24.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:24.983 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:24.983 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:24.984 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:24.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:24.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:24.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:24.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:45:24.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:45:24.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:24.986 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:24.986 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:45:24.986 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: test_get_osds: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:45:24.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-13T06:45:25.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:45:25.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:25.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:25.009 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:25.009 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:25.009 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:25.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:25.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-13T06:45:25.036 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:45:25.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:45:25.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:25.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:25.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:45:25.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:45:25.038 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:25.038 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:25.038 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:25.039 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:25.039 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:25.039 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:25.039 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:25.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:25.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:45:25.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:45:25.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:25.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:25.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:45:25.083 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:45:25.084 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:25.084 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:25.085 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:25.085 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:25.085 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:25.085 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:25.085 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:25.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:25.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:45:25.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1026: test_get_osds: run_mgr td/ceph-helpers x 2026-02-13T06:45:25.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:45:25.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:45:25.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:45:25.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:45:25.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:45:25.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:45:25.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:45:25.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:25.235 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:25.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:25.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:25.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:25.235 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:25.235 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:25.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:25.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: test_get_osds: run_osd td/ceph-helpers 0 2026-02-13T06:45:25.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:45:25.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:45:25.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:45:25.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:45:25.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:45:25.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:45:25.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:45:25.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:25.257 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:45:25.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:45:25.259 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:45:25.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=32e0dd23-1e5a-4d73-a137-c4fea51c9c4d 2026-02-13T06:45:25.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 32e0dd23-1e5a-4d73-a137-c4fea51c9c4d' 2026-02-13T06:45:25.259 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 32e0dd23-1e5a-4d73-a137-c4fea51c9c4d 2026-02-13T06:45:25.259 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:45:25.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCFyI5pg/XnDxAAfDpAUHiJgi48tEjNHDLKOw== 2026-02-13T06:45:25.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCFyI5pg/XnDxAAfDpAUHiJgi48tEjNHDLKOw=="}' 2026-02-13T06:45:25.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 32e0dd23-1e5a-4d73-a137-c4fea51c9c4d -i td/ceph-helpers/0/new.json 2026-02-13T06:45:25.373 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:25.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:45:25.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCFyI5pg/XnDxAAfDpAUHiJgi48tEjNHDLKOw== --osd-uuid 32e0dd23-1e5a-4d73-a137-c4fea51c9c4d 2026-02-13T06:45:25.398 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:25.397+0000 7f8205dc6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:25.400 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:25.398+0000 7f8205dc6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:25.400 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:25.399+0000 7f8205dc6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:25.401 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:25.400+0000 7f8205dc6900 -1 bdev(0x55e944b42c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:45:25.401 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:25.400+0000 7f8205dc6900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:45:25.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:45:25.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:45:25.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:45:25.806 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:45:25.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:45:25.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:45:25.912 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:45:25.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:45:25.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:45:25.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:45:25.913 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:45:25.929 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:25.928+0000 7fb0fe8d7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:25.931 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:25.929+0000 7fb0fe8d7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:25.932 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:25.930+0000 7fb0fe8d7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:26.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:45:26.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:45:26.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:45:26.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:45:26.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:45:26.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:26.016 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:26.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:45:26.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:26.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:26.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:45:26.153 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:26.152+0000 7fb0fe8d7900 -1 Falling back to public interface 2026-02-13T06:45:26.462 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:26.460+0000 7fb0fe8d7900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:45:27.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:45:27.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:27.114 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:27.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:45:27.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:27.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:27.215 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4047746526,v1:127.0.0.1:6801/4047746526] [v2:127.0.0.1:6802/4047746526,v1:127.0.0.1:6803/4047746526] exists,up 32e0dd23-1e5a-4d73-a137-c4fea51c9c4d 2026-02-13T06:45:27.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:45:27.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:45:27.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:45:27.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1028: test_get_osds: run_osd td/ceph-helpers 1 2026-02-13T06:45:27.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:45:27.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:45:27.220 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T06:45:27.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:45:27.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=115eed99-8ef0-45f5-9a8d-d7c5b0be5b1e 2026-02-13T06:45:27.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 115eed99-8ef0-45f5-9a8d-d7c5b0be5b1e' 2026-02-13T06:45:27.222 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 115eed99-8ef0-45f5-9a8d-d7c5b0be5b1e 2026-02-13T06:45:27.223 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:45:27.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCHyI5pgSO9DRAA63Fivvm7gffuq390eaZrfA== 2026-02-13T06:45:27.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCHyI5pgSO9DRAA63Fivvm7gffuq390eaZrfA=="}' 2026-02-13T06:45:27.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 115eed99-8ef0-45f5-9a8d-d7c5b0be5b1e -i td/ceph-helpers/1/new.json 2026-02-13T06:45:27.337 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:27.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T06:45:27.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCHyI5pgSO9DRAA63Fivvm7gffuq390eaZrfA== --osd-uuid 115eed99-8ef0-45f5-9a8d-d7c5b0be5b1e 2026-02-13T06:45:27.362 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:27.361+0000 7fe35387c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:27.363 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:27.362+0000 7fe35387c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:27.364 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:27.363+0000 7fe35387c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:27.365 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:27.363+0000 7fe35387c900 -1 bdev(0x55dffc0acc00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T06:45:27.365 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:27.363+0000 7fe35387c900 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T06:45:27.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T06:45:27.694 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:45:27.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T06:45:27.695 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T06:45:27.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:45:28.015 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T06:45:28.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T06:45:28.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:45:28.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:45:28.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:45:28.016 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:45:28.032 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:28.031+0000 7f70c7a0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:28.034 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:28.032+0000 7f70c7a0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:28.035 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:28.033+0000 7f70c7a0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:28.150 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:28.148+0000 7f70c7a0f900 -1 Falling back to public interface 2026-02-13T06:45:28.243 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T06:45:28.243 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:45:28.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T06:45:28.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:45:28.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:45:28.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:28.244 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:28.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:45:28.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:28.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:45:28.473 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:45:28.535 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:28.534+0000 7f70c7a0f900 -1 osd.1 0 log_to_monitors true 2026-02-13T06:45:29.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:45:29.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:29.475 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:29.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:45:29.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:29.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T06:45:29.700 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3186680752,v1:127.0.0.1:6811/3186680752] [v2:127.0.0.1:6812/3186680752,v1:127.0.0.1:6813/3186680752] exists,up 115eed99-8ef0-45f5-9a8d-d7c5b0be5b1e 2026-02-13T06:45:29.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:45:29.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:45:29.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:45:29.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1029: test_get_osds: create_rbd_pool 2026-02-13T06:45:29.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:45:29.932 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:45:29.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:45:29.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:45:30.215 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:45:30.223 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:45:31.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:45:31.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1030: test_get_osds: wait_for_clean 2026-02-13T06:45:31.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:45:31.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:45:31.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:45:31.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:45:31.509 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:31.509 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:31.509 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:31.509 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:31.509 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:31.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:45:31.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:45:31.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:45:31.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:45:31.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:45:31.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:45:31.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T06:45:31.819 INFO:tasks.workunit.client.0.trial040.stderr:1' 2026-02-13T06:45:31.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:45:31.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:45:31.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:45:31.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:45:31.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:45:31.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:45:31.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:45:31.884 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T06:45:31.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-13T06:45:31.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-13T06:45:31.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-42949672963' 2026-02-13T06:45:31.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:45:31.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:31.953 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:45:31.953 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:45:31.954 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:31.954 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:45:31.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:45:31.955 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:45:31.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:45:31.956 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:32.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:45:32.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:33.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:45:33.184 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:33.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-13T06:45:33.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:45:33.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-13T06:45:33.412 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:45:33.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T06:45:33.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672963 2026-02-13T06:45:33.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:45:33.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672963 2026-02-13T06:45:33.414 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672963 2026-02-13T06:45:33.415 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-13T06:45:33.415 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T06:45:33.640 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672963 -lt 42949672963 2026-02-13T06:45:33.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:45:33.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:33.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:33.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:45:33.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:45:33.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:45:33.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:45:33.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:45:33.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:45:33.953 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:45:33.953 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:45:34.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:45:34.180 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:45:34.180 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:34.180 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:34.494 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:45:34.494 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:45:34.494 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:45:34.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: test_get_osds: create_rbd_pool 2026-02-13T06:45:34.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:45:34.766 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:45:34.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:45:34.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:45:35.067 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:45:35.076 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:45:36.077 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:45:36.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: get_osds rbd GROUP 2026-02-13T06:45:36.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=rbd 2026-02-13T06:45:36.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=GROUP 2026-02-13T06:45:36.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: grep --quiet '^[0-1] [0-1]$' 2026-02-13T06:45:36.359 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map rbd GROUP 2026-02-13T06:45:36.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T06:45:36.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-13T06:45:36.591 INFO:tasks.workunit.client.0.trial040.stderr:0' 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1033: test_get_osds: teardown td/ceph-helpers 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:36.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:36.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:36.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:36.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:36.700 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:36.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:36.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:36.702 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:36.703 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:36.703 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:36.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:36.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:36.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:36.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:36.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:36.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:36.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:36.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:36.711 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:36.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:36.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:36.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:45:36.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_pg td/ceph-helpers 2026-02-13T06:45:36.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1091: test_get_pg: local dir=td/ceph-helpers 2026-02-13T06:45:36.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: test_get_pg: setup td/ceph-helpers 2026-02-13T06:45:36.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:45:36.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:45:36.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:36.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:36.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:36.714 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:36.714 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:36.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:36.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:36.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:36.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:36.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:36.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:36.716 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:36.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:36.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:36.717 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:36.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:36.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:36.718 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:36.718 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:36.718 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:36.719 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:36.719 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:36.721 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:36.721 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:36.721 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:36.721 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:36.722 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:36.722 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:36.722 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:45:36.723 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:45:36.723 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:36.723 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:36.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:45:36.724 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:45:36.725 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:45:36.725 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:45:36.725 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:45:36.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: test_get_pg: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:36.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:45:36.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:45:36.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:45:36.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:45:36.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:45:36.726 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:36.745 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:45:36.745 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:36.745 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:36.746 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:36.746 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:36.746 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:36.746 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:36.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:36.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:45:36.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:45:36.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:36.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:36.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:45:36.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:45:36.774 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:36.774 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:36.775 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:36.775 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:36.775 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:36.775 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:36.775 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:36.775 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:36.775 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:45:36.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:45:36.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:36.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:36.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:45:36.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:45:36.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:36.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:36.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:36.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:36.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:36.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:36.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:36.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:36.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:45:36.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1095: test_get_pg: run_mgr td/ceph-helpers x 2026-02-13T06:45:36.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:45:36.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:45:36.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:45:36.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:45:36.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:45:36.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:45:36.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:45:36.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:36.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:36.968 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:36.968 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:36.968 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:36.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:36.968 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:36.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:36.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: test_get_pg: run_osd td/ceph-helpers 0 2026-02-13T06:45:36.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:45:36.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:45:36.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:45:36.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:45:36.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:45:36.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:45:36.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:36.991 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:45:36.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:45:36.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:45:36.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:45:36.993 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:45:36.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d2133ede-a72f-4f89-ae27-20e4a9158437 2026-02-13T06:45:36.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d2133ede-a72f-4f89-ae27-20e4a9158437' 2026-02-13T06:45:36.994 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 d2133ede-a72f-4f89-ae27-20e4a9158437 2026-02-13T06:45:36.994 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:45:37.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCRyI5p/vcJABAApCRZynediQ43yQcVjwRufw== 2026-02-13T06:45:37.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCRyI5p/vcJABAApCRZynediQ43yQcVjwRufw=="}' 2026-02-13T06:45:37.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d2133ede-a72f-4f89-ae27-20e4a9158437 -i td/ceph-helpers/0/new.json 2026-02-13T06:45:37.110 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:37.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:45:37.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCRyI5p/vcJABAApCRZynediQ43yQcVjwRufw== --osd-uuid d2133ede-a72f-4f89-ae27-20e4a9158437 2026-02-13T06:45:37.136 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:37.134+0000 7f7494e53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:37.137 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:37.136+0000 7f7494e53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:37.138 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:37.137+0000 7f7494e53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:37.138 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:37.137+0000 7f7494e53900 -1 bdev(0x55daad8a4c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:45:37.138 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:37.137+0000 7f7494e53900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:45:37.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:45:37.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:45:37.559 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:45:37.559 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:45:37.559 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:45:37.664 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:45:37.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:45:37.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:45:37.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:45:37.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:45:37.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:45:37.682 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:37.681+0000 7f67fe272900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:37.683 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:37.682+0000 7f67fe272900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:37.685 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:37.683+0000 7f67fe272900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:37.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:45:37.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:45:37.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:45:37.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:45:37.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:45:37.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:37.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:45:37.768 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:37.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:37.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:37.867 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:45:37.870 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:37.868+0000 7f67fe272900 -1 Falling back to public interface 2026-02-13T06:45:38.254 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:38.253+0000 7f67fe272900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:45:38.868 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:45:38.868 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:38.868 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:38.869 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:45:38.869 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:38.869 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:38.967 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/188660240,v1:127.0.0.1:6801/188660240] [v2:127.0.0.1:6802/188660240,v1:127.0.0.1:6803/188660240] exists,up d2133ede-a72f-4f89-ae27-20e4a9158437 2026-02-13T06:45:38.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:45:38.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:45:38.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:45:38.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1097: test_get_pg: create_rbd_pool 2026-02-13T06:45:38.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:45:39.064 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:45:39.070 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:45:39.070 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:45:39.376 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:45:39.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:45:40.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:45:40.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1098: test_get_pg: wait_for_clean 2026-02-13T06:45:40.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:45:40.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:45:40.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:45:40.667 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:45:40.667 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:40.667 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:40.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:40.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:40.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:40.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:45:40.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:45:40.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:45:40.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:45:40.750 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:45:40.750 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:45:40.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:45:40.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:45:40.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:45:40.977 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:45:41.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:45:41.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:45:41.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:45:41.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:45:41.043 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:41.043 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:45:41.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:45:41.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:41.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:45:41.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:45:41.046 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:45:41.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:45:41.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:41.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:45:41.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:42.290 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:45:42.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:42.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:45:42.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:43.523 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T06:45:43.523 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:43.751 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:45:43.751 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:45:43.752 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:43.752 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:44.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:45:44.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:45:44.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:45:44.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:45:44.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:45:44.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:45:44.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:45:44.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:45:44.299 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:45:44.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:45:44.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:44.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:44.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:45:44.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:45:44.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:45:44.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: get_pg rbd GROUP 2026-02-13T06:45:44.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-13T06:45:44.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=GROUP 2026-02-13T06:45:44.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: grep --quiet '^[0-9]\.[0-9a-f][0-9a-f]*$' 2026-02-13T06:45:44.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd GROUP 2026-02-13T06:45:44.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-13T06:45:44.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1100: test_get_pg: teardown td/ceph-helpers 2026-02-13T06:45:44.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:44.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:44.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:44.837 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:44.837 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:44.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:44.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:44.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:44.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:44.943 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:44.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:44.944 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:44.946 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:44.946 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:44.946 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:44.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:44.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:44.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:44.948 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:44.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:44.949 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:44.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:44.953 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:44.953 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:44.953 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:44.953 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:44.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:44.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:44.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:45:44.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_primary td/ceph-helpers 2026-02-13T06:45:44.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: test_get_primary: local dir=td/ceph-helpers 2026-02-13T06:45:44.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: test_get_primary: setup td/ceph-helpers 2026-02-13T06:45:44.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:45:44.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:45:44.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:44.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:44.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:44.956 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:44.956 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:44.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:44.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:44.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:44.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:44.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:44.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:44.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:44.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:44.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:44.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:44.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:44.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:44.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:44.960 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:44.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:44.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:44.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:44.962 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:44.963 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:44.963 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:44.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:44.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:44.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:44.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:45:44.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:45:44.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:44.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:44.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:45:44.966 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:45:44.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:45:44.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:45:44.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:45:44.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: test_get_primary: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:44.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:45:44.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:45:44.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:45:44.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:45:44.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:45:44.968 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:44.988 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:45:44.988 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:44.988 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:44.988 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:44.988 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:44.988 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:44.988 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:44.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:45.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:45:45.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:45:45.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:45.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:45.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:45:45.016 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:45:45.017 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:45.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:45.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:45.018 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:45.018 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:45.018 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:45.018 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:45.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:45.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:45:45.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:45:45.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:45.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:45.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:45:45.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:45:45.063 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:45.063 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:45.063 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:45.063 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:45.063 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:45.063 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:45.063 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:45.063 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:45.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:45:45.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: test_get_primary: local osd=0 2026-02-13T06:45:45.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: test_get_primary: run_mgr td/ceph-helpers x 2026-02-13T06:45:45.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:45:45.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:45:45.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:45:45.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:45:45.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:45:45.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:45:45.209 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:45:45.209 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:45.209 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:45.209 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:45.209 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:45.210 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:45.210 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:45.210 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:45.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:45.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1201: test_get_primary: run_osd td/ceph-helpers 0 2026-02-13T06:45:45.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:45:45.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:45:45.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:45:45.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:45:45.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:45:45.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:45:45.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:45.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:45:45.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:45:45.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:45:45.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:45:45.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:45:45.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:45:45.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:45:45.236 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:45:45.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a1a43ab0-69d9-4885-b5fb-6050da4a5295 2026-02-13T06:45:45.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a1a43ab0-69d9-4885-b5fb-6050da4a5295' 2026-02-13T06:45:45.237 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 a1a43ab0-69d9-4885-b5fb-6050da4a5295 2026-02-13T06:45:45.237 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:45:45.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCZyI5pFaB6DhAA7zKrUNBB9MhhrndtK3F89Q== 2026-02-13T06:45:45.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCZyI5pFaB6DhAA7zKrUNBB9MhhrndtK3F89Q=="}' 2026-02-13T06:45:45.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a1a43ab0-69d9-4885-b5fb-6050da4a5295 -i td/ceph-helpers/0/new.json 2026-02-13T06:45:45.351 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:45.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:45:45.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCZyI5pFaB6DhAA7zKrUNBB9MhhrndtK3F89Q== --osd-uuid a1a43ab0-69d9-4885-b5fb-6050da4a5295 2026-02-13T06:45:45.377 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:45.375+0000 7f1a240ee900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:45.379 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:45.376+0000 7f1a240ee900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:45.380 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:45.377+0000 7f1a240ee900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:45.380 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:45.378+0000 7f1a240ee900 -1 bdev(0x564975578c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:45:45.380 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:45.378+0000 7f1a240ee900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:45:45.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:45:45.802 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:45:45.803 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:45:45.803 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:45:45.803 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:45:45.908 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:45:45.909 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:45:45.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:45:45.910 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:45:45.910 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:45:45.910 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:45:45.926 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:45.923+0000 7f9a74a69900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:45.927 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:45.925+0000 7f9a74a69900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:45.928 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:45.926+0000 7f9a74a69900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:46.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:45:46.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:45:46.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:45:46.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:45:46.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:45:46.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:46.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:45:46.011 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:46.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:46.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:46.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:45:46.150 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:46.148+0000 7f9a74a69900 -1 Falling back to public interface 2026-02-13T06:45:46.488 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:46.485+0000 7f9a74a69900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:45:47.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:45:47.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:47.112 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:47.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:45:47.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:47.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:47.213 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2309716414,v1:127.0.0.1:6801/2309716414] [v2:127.0.0.1:6802/2309716414,v1:127.0.0.1:6803/2309716414] exists,up a1a43ab0-69d9-4885-b5fb-6050da4a5295 2026-02-13T06:45:47.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:45:47.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:45:47.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:45:47.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: test_get_primary: create_rbd_pool 2026-02-13T06:45:47.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:45:47.312 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:45:47.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:45:47.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:45:47.561 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:45:47.568 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:45:48.570 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:45:48.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: test_get_primary: wait_for_clean 2026-02-13T06:45:48.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:45:48.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:45:48.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:45:48.851 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:45:48.851 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:48.851 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:48.851 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:48.851 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:48.851 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:48.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:45:48.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:45:48.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:45:48.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:45:48.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:45:48.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:45:49.161 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:45:49.161 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:45:49.161 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:45:49.161 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:45:49.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:45:49.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:45:49.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:45:49.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:45:49.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:49.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:45:49.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:45:49.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:49.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:45:49.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:45:49.229 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:45:49.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:45:49.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:49.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:45:49.458 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:45:49.458 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:49.458 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:49.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:45:49.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:45:49.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:45:49.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:45:49.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:45:49.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:45:49.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:45:49.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:45:49.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:45:49.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:45:49.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:49.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:50.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:45:50.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:45:50.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:45:50.310 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: get_primary rbd GROUP 2026-02-13T06:45:50.310 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-13T06:45:50.310 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-13T06:45:50.311 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-13T06:45:50.311 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-13T06:45:50.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: test 0 = 0 2026-02-13T06:45:50.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1205: test_get_primary: teardown td/ceph-helpers 2026-02-13T06:45:50.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:50.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:50.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:50.539 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:50.539 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:50.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:50.540 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:50.540 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:50.645 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:50.646 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:50.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:50.647 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:50.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:50.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:50.648 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:50.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:50.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:50.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:50.650 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:50.651 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:50.652 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:50.652 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:50.655 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:50.655 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:50.655 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:50.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:50.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:50.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:50.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:45:50.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_get_timeout_delays td/ceph-helpers 2026-02-13T06:45:50.657 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: get_timeout_delays 1 2026-02-13T06:45:50.657 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.657 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.657 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.657 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.657 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: test '1 ' = '1 ' 2026-02-13T06:45:50.667 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: get_timeout_delays 5 2026-02-13T06:45:50.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: test '1 2 2 ' = '1 2 2 ' 2026-02-13T06:45:50.686 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: get_timeout_delays 6 2026-02-13T06:45:50.687 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.687 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.687 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.687 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.687 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.703 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: test '1 2 3 ' = '1 2 3 ' 2026-02-13T06:45:50.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: get_timeout_delays 7 2026-02-13T06:45:50.705 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.705 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.705 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.705 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.705 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.727 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: test '1 2 4 ' = '1 2 4 ' 2026-02-13T06:45:50.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: get_timeout_delays 8 2026-02-13T06:45:50.729 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.729 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.751 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: test '1 2 4 1 ' = '1 2 4 1 ' 2026-02-13T06:45:50.753 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: get_timeout_delays 1 .1 2026-02-13T06:45:50.753 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.753 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.753 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.753 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.753 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: test '0.1 0.2 0.4 0.3 ' = '0.1 0.2 0.4 0.3 ' 2026-02-13T06:45:50.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: get_timeout_delays 1.5 .1 2026-02-13T06:45:50.777 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.777 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 ' = '0.1 0.2 0.4 0.8 ' 2026-02-13T06:45:50.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: get_timeout_delays 5 .1 2026-02-13T06:45:50.806 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.806 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.807 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.842 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 1.9 ' = '0.1 0.2 0.4 0.8 1.6 1.9 ' 2026-02-13T06:45:50.843 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: get_timeout_delays 6 .1 2026-02-13T06:45:50.844 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.844 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.844 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.844 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.844 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 2.9 ' = '0.1 0.2 0.4 0.8 1.6 2.9 ' 2026-02-13T06:45:50.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: get_timeout_delays 6.3 .1 2026-02-13T06:45:50.882 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.882 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 ' = '0.1 0.2 0.4 0.8 1.6 3.2 ' 2026-02-13T06:45:50.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: get_timeout_delays 20 .1 2026-02-13T06:45:50.919 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.919 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:50.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' 2026-02-13T06:45:50.970 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: get_timeout_delays 300 .1 0 2026-02-13T06:45:50.970 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:50.970 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:50.970 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:50.970 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:50.970 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:51.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' 2026-02-13T06:45:51.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: get_timeout_delays 300 .1 10 2026-02-13T06:45:51.046 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:51.046 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:51.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:51.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:51.046 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:51.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' 2026-02-13T06:45:51.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:45:51.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_is_clean td/ceph-helpers 2026-02-13T06:45:51.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1552: test_is_clean: local dir=td/ceph-helpers 2026-02-13T06:45:51.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1554: test_is_clean: setup td/ceph-helpers 2026-02-13T06:45:51.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:45:51.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:45:51.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:51.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:51.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:51.268 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:51.268 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:51.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:51.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:51.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:51.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:51.269 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:51.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:51.269 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:51.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:51.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:51.270 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:51.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:51.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:51.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:51.271 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:51.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:51.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:51.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:51.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:51.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:51.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:51.274 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:51.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:51.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:51.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:45:51.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:45:51.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:51.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:51.277 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:45:51.277 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:45:51.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:45:51.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:45:51.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:45:51.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1555: test_is_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:51.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:45:51.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:45:51.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:45:51.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:45:51.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:45:51.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:51.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:45:51.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:51.300 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:51.300 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:51.301 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:51.301 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:51.301 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:51.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:51.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:45:51.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:45:51.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:51.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:51.330 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:45:51.376 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:51.377 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:51.378 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:51.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:51.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:45:51.423 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1556: test_is_clean: run_mgr td/ceph-helpers x 2026-02-13T06:45:51.423 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:45:51.423 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:45:51.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:45:51.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:45:51.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:45:51.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:45:51.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:45:51.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:51.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:51.527 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:51.527 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:51.527 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:51.527 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:51.528 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:51.528 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:45:51.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1557: test_is_clean: run_osd td/ceph-helpers 0 2026-02-13T06:45:51.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:45:51.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:45:51.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:45:51.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:45:51.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:45:51.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:45:51.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:51.550 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:45:51.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:45:51.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:45:51.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:45:51.552 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:45:51.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b31050e8-294f-45be-8de6-c8c0e877e264 2026-02-13T06:45:51.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b31050e8-294f-45be-8de6-c8c0e877e264' 2026-02-13T06:45:51.553 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 b31050e8-294f-45be-8de6-c8c0e877e264 2026-02-13T06:45:51.553 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:45:51.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCfyI5ppehdIRAAD+OiED/6TxzM/e55D+dB/Q== 2026-02-13T06:45:51.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCfyI5ppehdIRAAD+OiED/6TxzM/e55D+dB/Q=="}' 2026-02-13T06:45:51.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b31050e8-294f-45be-8de6-c8c0e877e264 -i td/ceph-helpers/0/new.json 2026-02-13T06:45:51.667 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:51.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:45:51.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCfyI5ppehdIRAAD+OiED/6TxzM/e55D+dB/Q== --osd-uuid b31050e8-294f-45be-8de6-c8c0e877e264 2026-02-13T06:45:51.692 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:51.690+0000 7f5d9674b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:51.693 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:51.691+0000 7f5d9674b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:51.694 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:51.692+0000 7f5d9674b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:51.695 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:51.692+0000 7f5d9674b900 -1 bdev(0x557ea253cc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:45:51.695 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:51.692+0000 7f5d9674b900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:45:52.099 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:45:52.099 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:45:52.100 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:45:52.100 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:45:52.100 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:45:52.206 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:45:52.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:45:52.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:45:52.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:45:52.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:45:52.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:45:52.224 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:52.221+0000 7f22724b1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:52.225 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:52.223+0000 7f22724b1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:52.226 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:52.224+0000 7f22724b1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:45:52.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:45:52.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:45:52.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:45:52.310 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:45:52.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:45:52.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:52.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:45:52.311 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:45:52.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:52.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:52.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:45:52.420 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:52.417+0000 7f22724b1900 -1 Falling back to public interface 2026-02-13T06:45:52.720 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:45:52.718+0000 7f22724b1900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:45:53.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:45:53.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:45:53.412 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:45:53.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:45:53.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:45:53.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:45:53.514 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3579357715,v1:127.0.0.1:6801/3579357715] [v2:127.0.0.1:6802/3579357715,v1:127.0.0.1:6803/3579357715] exists,up b31050e8-294f-45be-8de6-c8c0e877e264 2026-02-13T06:45:53.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:45:53.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:45:53.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:45:53.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1558: test_is_clean: create_rbd_pool 2026-02-13T06:45:53.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:45:53.612 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:45:53.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:45:53.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:45:53.901 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:45:53.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:45:54.910 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:45:55.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1559: test_is_clean: wait_for_clean 2026-02-13T06:45:55.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:45:55.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:45:55.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:45:55.191 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:45:55.192 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:45:55.192 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:45:55.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:45:55.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:45:55.192 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:45:55.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:45:55.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:45:55.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:45:55.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:45:55.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:45:55.274 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:45:55.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:45:55.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:45:55.498 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:45:55.498 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:45:55.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:45:55.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:45:55.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:45:55.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:45:55.562 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:55.563 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:45:55.563 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:45:55.563 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:45:55.564 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:45:55.565 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:45:55.565 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:45:55.565 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:45:55.565 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:55.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:45:55.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:56.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:45:56.793 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:57.022 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:45:57.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:45:58.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T06:45:58.024 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:45:58.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:45:58.248 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:45:58.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:58.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:58.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:45:58.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:45:58.564 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:45:58.565 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:45:58.565 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:45:58.565 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:45:58.565 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:45:58.565 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:45:58.787 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:45:58.787 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:45:58.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:58.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:59.100 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:45:59.100 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:45:59.100 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:45:59.100 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1560: test_is_clean: is_clean 2026-02-13T06:45:59.100 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: get_num_pgs 2026-02-13T06:45:59.101 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:45:59.101 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:45:59.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: num_pgs=4 2026-02-13T06:45:59.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1547: is_clean: test 4 '!=' 0 2026-02-13T06:45:59.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: get_num_active_clean 2026-02-13T06:45:59.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:45:59.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:45:59.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:45:59.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:45:59.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:45:59.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: test 4 = 4 2026-02-13T06:45:59.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1561: test_is_clean: teardown td/ceph-helpers 2026-02-13T06:45:59.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:59.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:59.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:59.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:59.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:59.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:59.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:59.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:59.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:59.743 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:59.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:59.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:59.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:59.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:59.746 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:59.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:59.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:59.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:59.748 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:59.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:59.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:59.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:59.753 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:59.753 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:59.753 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:59.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:59.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:59.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:59.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:45:59.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_kill_daemon td/ceph-helpers 2026-02-13T06:45:59.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:266: test_kill_daemon: local dir=td/ceph-helpers 2026-02-13T06:45:59.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:267: test_kill_daemon: setup td/ceph-helpers 2026-02-13T06:45:59.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:45:59.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:45:59.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:45:59.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:45:59.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:45:59.756 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:45:59.756 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:45:59.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:45:59.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:45:59.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:45:59.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:45:59.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:45:59.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:45:59.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:45:59.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:45:59.758 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:45:59.759 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:45:59.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:59.759 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:45:59.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:45:59.760 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:45:59.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:45:59.761 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:45:59.761 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:45:59.762 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:45:59.763 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:59.763 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:59.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:45:59.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:45:59.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:45:59.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:45:59.764 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:45:59.765 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:59.765 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:59.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:45:59.766 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:45:59.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:45:59.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:45:59.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:45:59.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:268: test_kill_daemon: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:59.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:45:59.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:45:59.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:45:59.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:45:59.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:45:59.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:59.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:45:59.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:45:59.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:45:59.788 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:45:59.788 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:59.788 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:59.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:45:59.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:45:59.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:45:59.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:45:59.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:59.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:59.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:45:59.818 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:45:59.819 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:59.819 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:59.819 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:59.819 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:59.819 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:59.819 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:59.819 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:59.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:59.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:45:59.863 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:45:59.863 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:45:59.863 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:45:59.863 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:45:59.863 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:45:59.865 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:45:59.865 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:45:59.865 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:45:59.865 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:45:59.865 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:45:59.865 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:45:59.865 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:45:59.865 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:45:59.865 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:45:59.908 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:269: test_kill_daemon: run_mgr td/ceph-helpers x 2026-02-13T06:45:59.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:45:59.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:45:59.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:45:59.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:45:59.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:45:59.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:46:00.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:46:00.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:00.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:00.009 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:00.009 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:00.009 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:00.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:00.010 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:46:00.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:46:00.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:270: test_kill_daemon: run_osd td/ceph-helpers 0 2026-02-13T06:46:00.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:46:00.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:46:00.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:46:00.031 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:46:00.032 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:46:00.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f8ef883a-ebd6-4490-97db-8ba26d6a714d 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f8ef883a-ebd6-4490-97db-8ba26d6a714d' 2026-02-13T06:46:00.034 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 f8ef883a-ebd6-4490-97db-8ba26d6a714d 2026-02-13T06:46:00.035 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:46:00.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCoyI5pYZt8AhAA2qacKk61xYZFlP09yFmlyg== 2026-02-13T06:46:00.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCoyI5pYZt8AhAA2qacKk61xYZFlP09yFmlyg=="}' 2026-02-13T06:46:00.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f8ef883a-ebd6-4490-97db-8ba26d6a714d -i td/ceph-helpers/0/new.json 2026-02-13T06:46:00.148 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:00.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:46:00.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCoyI5pYZt8AhAA2qacKk61xYZFlP09yFmlyg== --osd-uuid f8ef883a-ebd6-4490-97db-8ba26d6a714d 2026-02-13T06:46:00.173 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:00.171+0000 7efe9ea0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:00.175 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:00.173+0000 7efe9ea0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:00.176 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:00.174+0000 7efe9ea0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:00.176 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:00.174+0000 7efe9ea0e900 -1 bdev(0x564bc367cc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:46:00.176 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:00.174+0000 7efe9ea0e900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:46:00.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:46:00.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:46:00.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:46:00.632 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:46:00.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:46:00.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:46:00.736 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:46:00.736 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:46:00.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:46:00.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:46:00.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:46:00.753 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:00.751+0000 7fc629c51900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:00.755 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:00.752+0000 7fc629c51900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:00.756 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:00.753+0000 7fc629c51900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:00.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:46:00.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:46:00.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:46:00.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:46:00.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:46:00.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:00.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:46:00.839 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:00.840 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:00.840 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:00.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:46:00.957 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:00.955+0000 7fc629c51900 -1 Falling back to public interface 2026-02-13T06:46:01.328 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:01.325+0000 7fc629c51900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:46:01.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:46:01.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:01.940 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:46:01.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:46:01.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:01.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:02.043 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3055217970,v1:127.0.0.1:6801/3055217970] [v2:127.0.0.1:6802/3055217970,v1:127.0.0.1:6803/3055217970] exists,up f8ef883a-ebd6-4490-97db-8ba26d6a714d 2026-02-13T06:46:02.043 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:46:02.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:46:02.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:46:02.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:272: test_kill_daemon: name_prefix=osd 2026-02-13T06:46:02.044 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: find td/ceph-helpers 2026-02-13T06:46:02.044 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: grep 'osd[^/]*\.pid' 2026-02-13T06:46:02.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-13T06:46:02.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:280: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid 0 1 2026-02-13T06:46:02.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-13T06:46:02.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=104088 2026-02-13T06:46:02.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=0 2026-02-13T06:46:02.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local delays=1 2026-02-13T06:46:02.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-13T06:46:02.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T06:46:02.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 104088 2026-02-13T06:46:02.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T06:46:02.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T06:46:02.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 1 2026-02-13T06:46:03.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T06:46:03.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 104088 2026-02-13T06:46:03.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T06:46:03.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T06:46:03.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0 2026-02-13T06:46:03.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 1 2026-02-13T06:46:03.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:284: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid TERM 2026-02-13T06:46:03.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-13T06:46:03.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=104088 2026-02-13T06:46:03.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-13T06:46:03.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-13T06:46:03.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-13T06:46:03.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T06:46:03.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 104088 2026-02-13T06:46:03.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T06:46:03.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T06:46:03.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-13T06:46:03.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T06:46:03.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 104088 2026-02-13T06:46:03.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-13T06:46:03.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-13T06:46:03.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-13T06:46:03.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:287: test_kill_daemon: name_prefix=mgr 2026-02-13T06:46:03.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: find td/ceph-helpers 2026-02-13T06:46:03.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: grep 'mgr[^/]*\.pid' 2026-02-13T06:46:03.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-13T06:46:03.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:292: test_kill_daemon: kill_daemon td/ceph-helpers/mgr.x.pid TERM 2026-02-13T06:46:03.153 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mgr.x.pid 2026-02-13T06:46:03.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=103233 2026-02-13T06:46:03.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-13T06:46:03.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-13T06:46:03.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-13T06:46:03.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T06:46:03.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 103233 2026-02-13T06:46:03.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T06:46:03.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T06:46:03.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-13T06:46:03.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T06:46:03.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 103233 2026-02-13T06:46:03.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-13T06:46:03.255 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-13T06:46:03.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-13T06:46:03.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:295: test_kill_daemon: name_prefix=mon 2026-02-13T06:46:03.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: find td/ceph-helpers 2026-02-13T06:46:03.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: grep 'mon[^/]*\.pid' 2026-02-13T06:46:03.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-13T06:46:03.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:300: test_kill_daemon: kill_daemon td/ceph-helpers/mon.a.pid TERM 2026-02-13T06:46:03.258 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mon.a.pid 2026-02-13T06:46:03.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=103148 2026-02-13T06:46:03.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-13T06:46:03.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-13T06:46:03.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-13T06:46:03.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T06:46:03.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 103148 2026-02-13T06:46:03.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T06:46:03.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T06:46:03.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-13T06:46:03.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T06:46:03.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 103148 2026-02-13T06:46:03.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-13T06:46:03.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-13T06:46:03.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-13T06:46:03.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:301: test_kill_daemon: timeout 5 ceph status 2026-02-13T06:46:08.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:304: test_kill_daemon: teardown td/ceph-helpers 2026-02-13T06:46:08.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:46:08.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:46:08.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:46:08.366 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:08.366 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:08.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:08.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:08.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:08.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:08.371 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:46:08.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:46:08.372 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:46:08.373 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:46:08.373 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:46:08.373 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:46:08.373 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:08.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:46:08.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:46:08.374 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:08.375 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:46:08.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:46:08.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:46:08.379 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:46:08.379 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:08.379 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:08.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:46:08.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:46:08.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:46:08.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:46:08.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_kill_daemons td/ceph-helpers 2026-02-13T06:46:08.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:357: test_kill_daemons: local dir=td/ceph-helpers 2026-02-13T06:46:08.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:358: test_kill_daemons: setup td/ceph-helpers 2026-02-13T06:46:08.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:46:08.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:46:08.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:46:08.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:46:08.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:46:08.382 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:08.382 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:08.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:08.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:08.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:08.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:08.383 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:46:08.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:46:08.383 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:46:08.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:46:08.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:46:08.384 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:46:08.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:08.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:46:08.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:46:08.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:08.386 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:46:08.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:46:08.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:46:08.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:46:08.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:08.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:08.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:46:08.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:46:08.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:46:08.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:46:08.390 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:46:08.390 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:08.391 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:08.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:46:08.391 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:359: test_kill_daemons: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:46:08.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:08.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:46:08.414 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:08.414 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:08.414 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:08.414 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:08.414 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:08.414 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:08.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:08.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:46:08.442 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:46:08.442 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:46:08.442 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:46:08.442 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:46:08.442 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:46:08.444 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:46:08.444 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:46:08.444 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:46:08.444 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:46:08.444 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:08.444 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:08.444 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:46:08.444 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:46:08.444 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:46:08.490 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:46:08.490 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:46:08.490 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:46:08.490 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:46:08.490 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:46:08.491 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:46:08.492 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:46:08.492 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:46:08.492 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:46:08.492 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:08.492 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:08.492 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:46:08.492 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:46:08.492 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:46:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:360: test_kill_daemons: run_mgr td/ceph-helpers x 2026-02-13T06:46:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:46:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:46:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:46:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:46:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:46:08.537 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:46:08.640 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:46:08.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:08.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:08.641 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:08.641 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:08.641 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:08.641 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:08.642 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:46:08.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:46:08.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:361: test_kill_daemons: run_osd td/ceph-helpers 0 2026-02-13T06:46:08.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:46:08.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:46:08.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:46:08.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:46:08.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:08.663 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:46:08.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:46:08.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:46:08.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:46:08.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:46:08.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:46:08.665 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:46:08.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=85972458-fadf-475b-a472-be7ae7cde57b 2026-02-13T06:46:08.666 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 85972458-fadf-475b-a472-be7ae7cde57b 2026-02-13T06:46:08.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 85972458-fadf-475b-a472-be7ae7cde57b' 2026-02-13T06:46:08.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:46:08.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCwyI5pTK4cKBAAsmp4f9wsYsMWe8FS6dTT7A== 2026-02-13T06:46:08.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCwyI5pTK4cKBAAsmp4f9wsYsMWe8FS6dTT7A=="}' 2026-02-13T06:46:08.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 85972458-fadf-475b-a472-be7ae7cde57b -i td/ceph-helpers/0/new.json 2026-02-13T06:46:08.779 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:08.786 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:46:08.787 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCwyI5pTK4cKBAAsmp4f9wsYsMWe8FS6dTT7A== --osd-uuid 85972458-fadf-475b-a472-be7ae7cde57b 2026-02-13T06:46:08.806 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:08.805+0000 7f3a9d3c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:08.808 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:08.806+0000 7f3a9d3c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:08.809 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:08.807+0000 7f3a9d3c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:08.809 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:08.808+0000 7f3a9d3c8900 -1 bdev(0x561fb7c1ec00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:46:08.809 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:08.808+0000 7f3a9d3c8900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:46:09.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:46:09.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:46:09.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:46:09.242 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:46:09.243 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:46:09.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:46:09.348 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:46:09.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:46:09.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:46:09.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:46:09.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:46:09.365 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:09.364+0000 7f98ee1a8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:09.367 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:09.366+0000 7f98ee1a8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:09.368 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:09.367+0000 7f98ee1a8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:09.452 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:46:09.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:46:09.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:46:09.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:46:09.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:46:09.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:09.453 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:09.454 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:46:09.454 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:09.454 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:09.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:46:09.570 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:09.569+0000 7f98ee1a8900 -1 Falling back to public interface 2026-02-13T06:46:09.993 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:09.991+0000 7f98ee1a8900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:46:10.553 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:46:10.553 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:10.554 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:46:10.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:46:10.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:10.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:10.657 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/22211462,v1:127.0.0.1:6801/22211462] [v2:127.0.0.1:6802/22211462,v1:127.0.0.1:6803/22211462] exists,up 85972458-fadf-475b-a472-be7ae7cde57b 2026-02-13T06:46:10.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:46:10.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:46:10.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:46:10.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:368: test_kill_daemons: kill_daemons td/ceph-helpers 0 osd 1 2026-02-13T06:46:10.659 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:10.659 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:10.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:10.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:10.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:11.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 1 2026-02-13T06:46:11.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:372: test_kill_daemons: kill_daemons td/ceph-helpers TERM osd 2026-02-13T06:46:11.665 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:11.665 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:11.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:11.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:11.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:11.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:11.971 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:376: test_kill_daemons: kill_daemons td/ceph-helpers TERM mgr 2026-02-13T06:46:11.971 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:11.972 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:11.972 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:11.972 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:11.972 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:12.078 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:12.078 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:380: test_kill_daemons: kill_daemons td/ceph-helpers TERM 2026-02-13T06:46:12.079 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:12.079 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:12.080 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:12.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:12.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:381: test_kill_daemons: timeout 5 ceph status 2026-02-13T06:46:17.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:382: test_kill_daemons: teardown td/ceph-helpers 2026-02-13T06:46:17.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:46:17.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:46:17.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:46:17.393 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:17.393 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:17.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:17.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:17.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:17.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:17.397 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:46:17.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:46:17.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:46:17.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:46:17.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:46:17.400 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:46:17.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:17.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:46:17.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:46:17.401 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:17.401 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:46:17.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:46:17.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:46:17.405 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:46:17.405 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:17.406 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:17.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:46:17.407 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_objectstore_tool td/ceph-helpers 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: test_objectstore_tool: local dir=td/ceph-helpers 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: test_objectstore_tool: setup td/ceph-helpers 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:46:17.408 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:17.409 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:17.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:17.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:17.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:17.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:17.409 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:46:17.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:46:17.410 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:46:17.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:46:17.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:46:17.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:46:17.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:17.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:46:17.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:46:17.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:17.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:46:17.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:46:17.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:46:17.415 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:46:17.415 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:17.415 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:17.416 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:46:17.416 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:46:17.416 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:46:17.416 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:46:17.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:46:17.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:17.418 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:17.418 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:46:17.418 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:46:17.419 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:46:17.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:46:17.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:46:17.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1301: test_objectstore_tool: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:17.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:46:17.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:46:17.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:46:17.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:46:17.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:46:17.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:17.440 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:46:17.440 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:17.440 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:17.440 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:17.440 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:17.440 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:17.441 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:17.441 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:17.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:46:17.470 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:46:17.470 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:46:17.470 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:46:17.470 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:46:17.471 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:46:17.471 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:46:17.471 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:46:17.471 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:46:17.471 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:46:17.471 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:17.471 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:17.472 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:46:17.472 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:46:17.472 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:46:17.517 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:46:17.517 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:46:17.517 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:46:17.517 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:46:17.517 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:46:17.518 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:46:17.518 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:46:17.518 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:46:17.519 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:46:17.519 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:17.519 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:17.519 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:46:17.519 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:46:17.519 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:46:17.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: test_objectstore_tool: local osd=0 2026-02-13T06:46:17.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: test_objectstore_tool: run_mgr td/ceph-helpers x 2026-02-13T06:46:17.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:46:17.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:46:17.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:46:17.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:46:17.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:46:17.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:46:17.664 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:46:17.665 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:17.665 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:17.665 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:17.666 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:17.666 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:17.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:17.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:46:17.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:46:17.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: test_objectstore_tool: run_osd td/ceph-helpers 0 2026-02-13T06:46:17.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:46:17.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:46:17.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:46:17.687 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:46:17.688 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:46:17.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:46:17.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:46:17.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:46:17.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:46:17.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:46:17.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:46:17.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:46:17.690 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:46:17.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b736d3a5-d65f-4954-9895-172b73bd5222 2026-02-13T06:46:17.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b736d3a5-d65f-4954-9895-172b73bd5222' 2026-02-13T06:46:17.691 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 b736d3a5-d65f-4954-9895-172b73bd5222 2026-02-13T06:46:17.691 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:46:17.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC5yI5p7TWcKRAA5KFFBGW4eTbn0Ci9o96/AQ== 2026-02-13T06:46:17.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC5yI5p7TWcKRAA5KFFBGW4eTbn0Ci9o96/AQ=="}' 2026-02-13T06:46:17.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b736d3a5-d65f-4954-9895-172b73bd5222 -i td/ceph-helpers/0/new.json 2026-02-13T06:46:17.802 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:17.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:46:17.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC5yI5p7TWcKRAA5KFFBGW4eTbn0Ci9o96/AQ== --osd-uuid b736d3a5-d65f-4954-9895-172b73bd5222 2026-02-13T06:46:17.827 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:17.825+0000 7f029ee53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:17.828 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:17.827+0000 7f029ee53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:17.829 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:17.828+0000 7f029ee53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:17.829 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:17.828+0000 7f029ee53900 -1 bdev(0x559770d86c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:46:17.830 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:17.828+0000 7f029ee53900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:46:18.282 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:46:18.282 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:46:18.283 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:46:18.283 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:46:18.283 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:46:18.387 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:46:18.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:46:18.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:46:18.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:46:18.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:46:18.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:46:18.404 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:18.403+0000 7f9874f8a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:18.405 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:18.404+0000 7f9874f8a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:18.406 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:18.405+0000 7f9874f8a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:18.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:46:18.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:46:18.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:46:18.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:46:18.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:46:18.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:18.489 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:46:18.490 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:18.490 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:18.490 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:18.585 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:18.584+0000 7f9874f8a900 -1 Falling back to public interface 2026-02-13T06:46:18.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:46:18.902 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:18.900+0000 7f9874f8a900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:46:19.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:46:19.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:19.589 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:46:19.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:46:19.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:19.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:19.689 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2785831838,v1:127.0.0.1:6801/2785831838] [v2:127.0.0.1:6802/2785831838,v1:127.0.0.1:6803/2785831838] exists,up b736d3a5-d65f-4954-9895-172b73bd5222 2026-02-13T06:46:19.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:46:19.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:46:19.691 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:46:19.691 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: test_objectstore_tool: create_rbd_pool 2026-02-13T06:46:19.691 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:46:19.790 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:46:19.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:46:19.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:46:20.078 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:46:20.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:46:21.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:46:21.368 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: test_objectstore_tool: wait_for_clean 2026-02-13T06:46:21.369 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:46:21.369 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:46:21.369 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:46:21.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:46:21.370 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:46:21.370 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:46:21.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:46:21.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:46:21.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:46:21.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:46:21.452 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:46:21.452 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:46:21.452 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:46:21.452 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:46:21.452 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:46:21.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:46:21.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:46:21.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:46:21.678 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:46:21.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:46:21.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:46:21.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:46:21.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:46:21.745 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:46:21.745 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:46:21.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:46:21.746 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:46:21.746 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:46:21.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:46:21.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:46:21.748 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:46:21.748 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:21.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:46:21.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:46:22.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:46:22.975 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:23.201 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T06:46:23.201 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:46:24.202 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T06:46:24.203 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:24.427 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:46:24.428 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:46:24.428 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:46:24.428 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:46:24.743 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:46:24.744 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:46:24.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:46:24.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:46:24.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:46:24.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:46:24.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:46:24.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:46:24.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:46:24.969 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:46:24.970 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:46:24.970 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:46:25.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:46:25.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:46:25.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:46:25.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1307: test_objectstore_tool: rados --pool rbd put GROUP /etc/group 2026-02-13T06:46:25.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1308: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 GROUP get-bytes 2026-02-13T06:46:25.300 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 GROUP get-bytes 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1309: test_objectstore_tool: diff - /etc/group 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:25.301 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:25.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:25.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:25.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:25.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:25.405 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 GROUP get-bytes 2026-02-13T06:46:25.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-13T06:46:25.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T06:46:25.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-13T06:46:25.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T06:46:25.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-13T06:46:25.406 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 GROUP get-bytes 2026-02-13T06:46:25.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/ceph-helpers 0 2026-02-13T06:46:25.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T06:46:25.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T06:46:25.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:46:25.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:start osd.0 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-13T06:46:25.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T06:46:25.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T06:46:25.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T06:46:25.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T06:46:25.520 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:25.519+0000 7f479d223900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:25.522 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:25.521+0000 7f479d223900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:25.523 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:25.522+0000 7f479d223900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:25.656 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:25.655+0000 7f479d223900 -1 Falling back to public interface 2026-02-13T06:46:25.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T06:46:25.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:46:25.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:46:25.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:46:25.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:46:25.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:25.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:46:25.733 INFO:tasks.workunit.client.0.trial040.stderr:0 2026-02-13T06:46:25.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:25.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:25.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:46:25.996 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:25.995+0000 7f479d223900 -1 osd.0 12 log_to_monitors true 2026-02-13T06:46:26.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:46:26.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:26.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:46:26.964 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T06:46:26.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:26.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:27.190 INFO:tasks.workunit.client.0.trial040.stderr:osd.0 up in weight 1 up_from 15 up_thru 15 down_at 12 last_clean_interval [5,12) [v2:127.0.0.1:6800/2922838334,v1:127.0.0.1:6801/2922838334] [v2:127.0.0.1:6802/2922838334,v1:127.0.0.1:6803/2922838334] exists,up b736d3a5-d65f-4954-9895-172b73bd5222 2026-02-13T06:46:27.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:46:27.190 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:46:27.191 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:46:27.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:46:27.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:46:27.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:46:27.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:46:27.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:46:27.269 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:46:27.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:46:27.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:46:27.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:46:27.495 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:46:27.559 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-13T06:46:27.559 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-13T06:46:27.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-64424509443' 2026-02-13T06:46:27.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:46:27.560 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-64424509443 2026-02-13T06:46:27.560 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:46:27.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:46:27.561 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-64424509443 2026-02-13T06:46:27.561 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:46:27.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509443 2026-02-13T06:46:27.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 64424509443' 2026-02-13T06:46:27.562 INFO:tasks.workunit.client.0.trial040.stderr:waiting osd.0 seq 64424509443 2026-02-13T06:46:27.562 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:27.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-13T06:46:27.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:46:27.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:46:27.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:46:28.100 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:46:28.100 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:46:28.100 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:46:28.100 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:46:28.100 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:46:28.100 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:46:28.101 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:46:28.101 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:46:28.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:46:28.326 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:46:28.326 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:46:28.327 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:46:28.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:46:28.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:46:28.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:46:28.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1310: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:28.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:28.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:28.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:28.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:28.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:28.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-13T06:46:28.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-13T06:46:28.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T06:46:28.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-13T06:46:28.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T06:46:28.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-13T06:46:28.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 NOTEXISTS get-bytes 2026-02-13T06:46:28.790 INFO:tasks.workunit.client.0.trial040.stderr:No object id 'NOTEXISTS' found or invalid JSON specified 2026-02-13T06:46:28.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: return 1 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: return 1 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1311: test_objectstore_tool: teardown td/ceph-helpers 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:28.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:28.832 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:28.936 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:28.937 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:46:28.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:46:28.938 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:46:28.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:46:28.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:46:28.940 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:46:28.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:28.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:46:28.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:46:28.941 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:28.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:46:28.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:46:28.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:46:28.947 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:46:28.947 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:28.947 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:28.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:46:28.949 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:46:28.949 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_pg_schedule_scrub td/ceph-helpers 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1987: test_pg_schedule_scrub: local dir=td/ceph-helpers 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1989: test_pg_schedule_scrub: setup td/ceph-helpers 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:28.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:28.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:28.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:28.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:28.951 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:46:28.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:46:28.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:46:28.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:46:28.953 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:46:28.953 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:46:28.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:28.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:46:28.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:46:28.954 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:28.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:46:28.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:46:28.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:46:28.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:46:28.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:28.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:28.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:46:28.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:46:28.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:46:28.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:46:28.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:46:28.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:28.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:28.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:46:28.961 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:46:28.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:46:28.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:46:28.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:46:28.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1990: test_pg_schedule_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:28.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:46:28.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:46:28.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:46:28.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:46:28.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:46:28.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:28.982 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:46:28.982 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:28.982 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:28.982 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:28.983 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:28.983 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:28.983 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:28.983 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:29.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:46:29.012 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:46:29.012 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:46:29.012 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:46:29.012 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:46:29.013 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:46:29.014 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:46:29.014 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:46:29.014 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:46:29.014 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:46:29.014 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:29.014 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:29.014 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:46:29.014 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:46:29.015 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:46:29.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:46:29.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:46:29.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:46:29.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:46:29.058 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:46:29.058 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:46:29.058 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:46:29.058 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:46:29.059 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:46:29.059 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:29.059 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:29.059 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:46:29.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:46:29.059 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:46:29.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1991: test_pg_schedule_scrub: run_mgr td/ceph-helpers x --mgr_stats_period=1 2026-02-13T06:46:29.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:46:29.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:46:29.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:46:29.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:46:29.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:46:29.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:46:29.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:46:29.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:29.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:29.205 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:29.205 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:29.205 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:29.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:29.206 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:46:29.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-13T06:46:29.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1992: test_pg_schedule_scrub: run_osd td/ceph-helpers 0 2026-02-13T06:46:29.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:46:29.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:46:29.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:46:29.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:46:29.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:46:29.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:46:29.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:46:29.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:46:29.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:46:29.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:46:29.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:46:29.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:46:29.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:46:29.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=55b2bc41-f2b0-4a25-94cc-92827db8214b 2026-02-13T06:46:29.231 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 55b2bc41-f2b0-4a25-94cc-92827db8214b 2026-02-13T06:46:29.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 55b2bc41-f2b0-4a25-94cc-92827db8214b' 2026-02-13T06:46:29.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:46:29.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDFyI5prcsvDhAAtDwii12mJ57AcKh2eGaA2g== 2026-02-13T06:46:29.240 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDFyI5prcsvDhAAtDwii12mJ57AcKh2eGaA2g=="}' 2026-02-13T06:46:29.240 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 55b2bc41-f2b0-4a25-94cc-92827db8214b -i td/ceph-helpers/0/new.json 2026-02-13T06:46:29.342 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:29.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:46:29.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDFyI5prcsvDhAAtDwii12mJ57AcKh2eGaA2g== --osd-uuid 55b2bc41-f2b0-4a25-94cc-92827db8214b 2026-02-13T06:46:29.368 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:29.367+0000 7f8e49853900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:29.370 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:29.369+0000 7f8e49853900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:29.371 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:29.370+0000 7f8e49853900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:29.371 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:29.370+0000 7f8e49853900 -1 bdev(0x557015168c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:46:29.372 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:29.370+0000 7f8e49853900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:46:29.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:46:29.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:46:29.769 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:46:29.769 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:46:29.769 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:46:29.875 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:46:29.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:46:29.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:46:29.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:46:29.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:46:29.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:46:29.892 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:29.891+0000 7f98ccc10900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:29.894 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:29.893+0000 7f98ccc10900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:29.895 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:29.894+0000 7f98ccc10900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:29.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:46:29.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:46:29.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:46:29.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:46:29.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:46:29.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:29.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:46:29.979 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:29.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:29.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:30.081 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:46:30.107 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:30.105+0000 7f98ccc10900 -1 Falling back to public interface 2026-02-13T06:46:30.411 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:30.410+0000 7f98ccc10900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:46:31.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:46:31.083 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:31.083 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:46:31.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:46:31.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:31.084 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:31.183 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/760506543,v1:127.0.0.1:6801/760506543] [v2:127.0.0.1:6802/760506543,v1:127.0.0.1:6803/760506543] exists,up 55b2bc41-f2b0-4a25-94cc-92827db8214b 2026-02-13T06:46:31.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:46:31.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:46:31.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:46:31.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1993: test_pg_schedule_scrub: create_rbd_pool 2026-02-13T06:46:31.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:46:31.279 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:46:31.285 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:46:31.285 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:46:31.596 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:46:31.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:46:32.605 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:46:32.888 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1994: test_pg_schedule_scrub: wait_for_clean 2026-02-13T06:46:32.888 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:46:32.888 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:46:32.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:46:32.889 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:46:32.889 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:46:32.889 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:46:32.889 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:46:32.889 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:46:32.889 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:46:32.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:46:32.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:46:32.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:46:32.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:46:32.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:46:32.970 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:46:33.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:46:33.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:46:33.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:46:33.194 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:46:33.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T06:46:33.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T06:46:33.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T06:46:33.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:46:33.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T06:46:33.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:46:33.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:46:33.261 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-13T06:46:33.261 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:46:33.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-13T06:46:33.263 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T06:46:33.263 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836484 2026-02-13T06:46:33.263 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:33.494 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T06:46:33.494 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:46:34.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:46:34.495 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:34.719 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T06:46:34.719 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:46:35.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T06:46:35.721 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:35.949 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836484 2026-02-13T06:46:35.949 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:46:35.949 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:46:35.950 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:46:36.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:46:36.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:46:36.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:46:36.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:46:36.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:46:36.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:46:36.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:46:36.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:46:36.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:46:36.502 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:46:36.502 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:46:36.502 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:46:36.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1995: test_pg_schedule_scrub: pg_schedule_scrub 1.0 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1969: pg_schedule_scrub: local pgid=1.0 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1971: pg_schedule_scrub: wait_for_pg_clean 1.0 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:46:36.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:46:36.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:46:36.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:46:37.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-13T06:46:37.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-13T06:46:37.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-13T06:46:37.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-13T06:46:37.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:46:37.015 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:46:37.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:46:37.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:46:37.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:46:37.244 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:46:37.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-13T06:46:37.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-13T06:46:37.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490' 2026-02-13T06:46:37.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:46:37.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-13T06:46:37.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:46:37.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:46:37.313 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836490 2026-02-13T06:46:37.313 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:46:37.314 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836490 2026-02-13T06:46:37.315 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-13T06:46:37.315 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836490 2026-02-13T06:46:37.315 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:37.542 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T06:46:37.542 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:46:38.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:46:38.544 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:38.771 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T06:46:38.771 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:46:39.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T06:46:39.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:40.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836490 2026-02-13T06:46:40.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T06:46:40.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-13T06:46:40.003 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 0 2026-02-13T06:46:40.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T06:46:40.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T06:46:40.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T06:46:40.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T06:46:40.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T06:46:40.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-13T06:46:40.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T06:46:40.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-13T06:46:40.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-13T06:46:40.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: get_last_scrub_stamp 1.0 2026-02-13T06:46:40.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:40.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:40.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:40.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:40.297 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: local last_scrub=2026-02-13T06:46:31.529283+0000 2026-02-13T06:46:40.297 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1973: pg_schedule_scrub: ceph tell 1.0 schedule-scrub 2026-02-13T06:46:40.361 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T06:46:40.361 INFO:tasks.workunit.client.0.trial040.stdout: "deep": false, 2026-02-13T06:46:40.361 INFO:tasks.workunit.client.0.trial040.stdout: "must": false, 2026-02-13T06:46:40.361 INFO:tasks.workunit.client.0.trial040.stdout: "stamp": "2026-02-10T18:46:40.360426+0000" 2026-02-13T06:46:40.361 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T06:46:40.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1974: pg_schedule_scrub: wait_for_scrub 1.0 2026-02-13T06:46:31.529283+0000 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-13T06:46:31.529283+0000 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:40.367 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:40.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:31.529283+0000 '>' 2026-02-13T06:46:31.529283+0000 2026-02-13T06:46:40.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T06:46:41.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T06:46:41.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:41.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:41.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:41.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:41.592 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:41.592 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:41.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:31.529283+0000 '>' 2026-02-13T06:46:31.529283+0000 2026-02-13T06:46:41.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T06:46:42.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T06:46:42.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:42.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:42.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:42.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:42.817 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:42.818 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:43.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:31.529283+0000 '>' 2026-02-13T06:46:31.529283+0000 2026-02-13T06:46:43.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T06:46:44.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T06:46:44.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:44.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:44.048 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:44.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:44.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:44.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:44.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:41.429930+0000 '>' 2026-02-13T06:46:31.529283+0000 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1996: test_pg_schedule_scrub: teardown td/ceph-helpers 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:44.272 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:44.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:44.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:46:44.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:46:44.379 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:46:44.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:46:44.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:46:44.381 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:46:44.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:44.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:46:44.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:46:44.383 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:44.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:46:44.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:46:44.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:46:44.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:46:44.389 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:44.389 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:44.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_pg_scrub td/ceph-helpers 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: test_pg_scrub: local dir=td/ceph-helpers 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: test_pg_scrub: setup td/ceph-helpers 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T06:46:44.391 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T06:46:44.392 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:44.392 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:44.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:44.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:44.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:44.392 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:44.393 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T06:46:44.393 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T06:46:44.393 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T06:46:44.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T06:46:44.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T06:46:44.394 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T06:46:44.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:44.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T06:46:44.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T06:46:44.395 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T06:46:44.396 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T06:46:44.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T06:46:44.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T06:46:44.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T06:46:44.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:44.398 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:44.398 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T06:46:44.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T06:46:44.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T06:46:44.399 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T06:46:44.400 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T06:46:44.400 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:44.400 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:44.400 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T06:46:44.401 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T06:46:44.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T06:46:44.402 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T06:46:44.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T06:46:44.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: test_pg_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:44.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T06:46:44.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T06:46:44.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T06:46:44.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T06:46:44.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T06:46:44.403 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:44.423 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T06:46:44.424 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:44.424 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:44.424 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:44.424 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:44.424 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:44.424 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:44.424 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T06:46:44.452 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T06:46:44.453 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T06:46:44.453 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:46:44.453 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:46:44.453 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T06:46:44.454 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T06:46:44.454 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:46:44.454 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:46:44.454 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:46:44.455 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:46:44.455 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:44.455 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:44.455 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:46:44.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:46:44.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T06:46:44.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T06:46:44.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T06:46:44.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T06:46:44.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T06:46:44.500 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T06:46:44.501 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T06:46:44.501 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T06:46:44.501 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T06:46:44.502 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T06:46:44.502 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:44.502 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:44.502 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T06:46:44.502 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T06:46:44.502 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T06:46:44.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: test_pg_scrub: run_mgr td/ceph-helpers x 2026-02-13T06:46:44.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T06:46:44.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T06:46:44.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T06:46:44.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T06:46:44.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T06:46:44.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T06:46:44.648 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T06:46:44.648 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:44.648 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:44.649 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:44.649 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:44.649 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:44.649 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:44.649 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:46:44.649 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T06:46:44.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: test_pg_scrub: run_osd td/ceph-helpers 0 2026-02-13T06:46:44.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T06:46:44.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T06:46:44.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T06:46:44.670 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T06:46:44.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T06:46:44.672 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T06:46:44.672 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T06:46:44.672 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T06:46:44.672 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T06:46:44.672 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T06:46:44.672 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T06:46:44.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=15e2566a-9217-4e72-b873-28f8820db63c 2026-02-13T06:46:44.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 15e2566a-9217-4e72-b873-28f8820db63c' 2026-02-13T06:46:44.673 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 15e2566a-9217-4e72-b873-28f8820db63c 2026-02-13T06:46:44.673 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T06:46:44.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDUyI5plE+cKBAAICgtfh5t98ArIDbH4Xo98Q== 2026-02-13T06:46:44.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDUyI5plE+cKBAAICgtfh5t98ArIDbH4Xo98Q=="}' 2026-02-13T06:46:44.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 15e2566a-9217-4e72-b873-28f8820db63c -i td/ceph-helpers/0/new.json 2026-02-13T06:46:44.789 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:44.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T06:46:44.797 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDUyI5plE+cKBAAICgtfh5t98ArIDbH4Xo98Q== --osd-uuid 15e2566a-9217-4e72-b873-28f8820db63c 2026-02-13T06:46:44.816 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:44.813+0000 7f221fd8a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:44.817 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:44.815+0000 7f221fd8a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:44.818 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:44.816+0000 7f221fd8a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:44.819 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:44.816+0000 7f221fd8a900 -1 bdev(0x564b178e0c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T06:46:44.819 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:44.816+0000 7f221fd8a900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T06:46:45.240 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T06:46:45.240 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T06:46:45.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T06:46:45.241 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T06:46:45.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T06:46:45.349 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T06:46:45.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T06:46:45.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T06:46:45.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T06:46:45.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T06:46:45.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T06:46:45.366 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:45.364+0000 7f1a178e7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:45.367 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:45.365+0000 7f1a178e7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:45.368 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:45.366+0000 7f1a178e7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T06:46:45.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T06:46:45.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T06:46:45.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T06:46:45.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T06:46:45.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T06:46:45.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:45.453 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T06:46:45.454 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T06:46:45.454 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:45.454 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:45.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T06:46:45.622 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:45.620+0000 7f1a178e7900 -1 Falling back to public interface 2026-02-13T06:46:46.036 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T06:46:46.034+0000 7f1a178e7900 -1 osd.0 0 log_to_monitors true 2026-02-13T06:46:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T06:46:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T06:46:46.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T06:46:46.556 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T06:46:46.556 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T06:46:46.556 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T06:46:46.657 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1782860686,v1:127.0.0.1:6801/1782860686] [v2:127.0.0.1:6802/1782860686,v1:127.0.0.1:6803/1782860686] exists,up 15e2566a-9217-4e72-b873-28f8820db63c 2026-02-13T06:46:46.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T06:46:46.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T06:46:46.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T06:46:46.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1947: test_pg_scrub: create_rbd_pool 2026-02-13T06:46:46.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T06:46:46.756 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T06:46:46.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T06:46:46.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T06:46:47.006 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T06:46:47.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T06:46:48.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T06:46:48.297 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1948: test_pg_scrub: wait_for_clean 2026-02-13T06:46:48.298 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T06:46:48.298 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T06:46:48.298 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T06:46:48.299 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T06:46:48.299 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:46:48.299 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:46:48.299 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:46:48.299 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:46:48.299 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:46:48.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T06:46:48.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T06:46:48.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T06:46:48.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T06:46:48.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:46:48.380 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:46:48.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:46:48.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:46:48.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:46:48.608 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:46:48.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T06:46:48.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T06:46:48.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T06:46:48.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:46:48.675 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T06:46:48.675 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:46:48.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:46:48.676 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T06:46:48.676 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:46:48.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T06:46:48.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T06:46:48.678 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T06:46:48.678 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:48.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T06:46:48.904 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T06:46:48.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:46:48.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:46:49.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T06:46:49.221 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T06:46:49.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T06:46:49.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T06:46:49.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T06:46:49.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T06:46:49.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T06:46:49.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T06:46:49.445 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T06:46:49.445 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T06:46:49.446 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T06:46:49.446 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1949: test_pg_scrub: pg_scrub 1.0 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:46:49.767 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:46:49.768 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:46:49.768 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:46:49.768 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:46:49.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-13T06:46:49.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-13T06:46:49.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-13T06:46:49.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-13T06:46:49.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T06:46:49.967 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:46:50.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:46:50.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:46:50.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:46:50.194 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T06:46:50.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-13T06:46:50.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-13T06:46:50.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-13T06:46:50.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T06:46:50.259 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-13T06:46:50.259 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T06:46:50.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T06:46:50.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-13T06:46:50.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T06:46:50.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-13T06:46:50.262 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-13T06:46:50.262 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836485 2026-02-13T06:46:50.262 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:50.492 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-13T06:46:50.492 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T06:46:51.493 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T06:46:51.494 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T06:46:51.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-13T06:46:51.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T06:46:51.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-13T06:46:51.732 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 0 2026-02-13T06:46:51.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T06:46:51.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T06:46:51.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T06:46:51.733 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T06:46:51.733 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T06:46:51.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-13T06:46:51.802 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T06:46:51.802 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-13T06:46:51.802 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-13T06:46:51.802 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-13T06:46:51.802 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:51.802 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:51.802 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:51.802 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:52.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:52.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-13T06:46:52.241 INFO:tasks.workunit.client.0.trial040.stderr:instructing pg 1.0 on osd.0 to scrub 2026-02-13T06:46:52.249 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:52.249 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-13T06:46:52.249 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:52.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-13T06:46:52.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-13T06:46:52.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:52.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:52.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:52.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:52.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:52.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:52.472 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:46.942699+0000 '>' 2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:52.472 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T06:46:53.473 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T06:46:53.473 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:53.474 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:53.474 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:53.474 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:53.474 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:53.474 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:53.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:46.942699+0000 '>' 2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:53.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T06:46:54.698 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T06:46:54.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:54.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:54.700 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:54.700 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:54.700 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:54.700 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:54.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:46.942699+0000 '>' 2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:54.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T06:46:55.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T06:46:55.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:55.923 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:55.923 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:55.923 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:55.923 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:55.924 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:56.156 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:46.942699+0000 '>' 2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:56.156 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T06:46:57.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T06:46:57.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:57.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:57.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:57.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:57.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:57.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:57.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:46.942699+0000 '>' 2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:57.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T06:46:58.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T06:46:58.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:58.385 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:58.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:58.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:58.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:58.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:58.610 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:46.942699+0000 '>' 2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:58.610 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T06:46:59.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T06:46:59.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T06:46:59.612 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T06:46:59.612 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T06:46:59.612 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T06:46:59.612 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T06:46:59.612 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T06:46:59.836 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T06:46:52.954934+0000 '>' 2026-02-13T06:46:46.942699+0000 2026-02-13T06:46:59.836 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-13T06:46:59.836 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1950: test_pg_scrub: kill_daemons td/ceph-helpers KILL osd 2026-02-13T06:46:59.836 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T06:46:59.836 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T06:46:59.836 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T06:46:59.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T06:46:59.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T06:46:59.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T06:46:59.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: TIMEOUT=2 2026-02-13T06:46:59.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: pg_scrub 1.0 2026-02-13T06:46:59.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-13T06:46:59.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-13T06:46:59.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-13T06:46:59.943 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-13T06:46:59.943 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T06:46:59.944 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T06:46:59.944 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T06:46:59.944 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T06:46:59.944 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T06:47:00.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-13T06:47:00.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-13T06:47:00.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-13T06:47:00.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-13T06:47:00.143 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=2 2026-02-13T06:47:00.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T06:47:00.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T06:47:00.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T06:47:00.375 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T06:47:00.375 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:01:49.486 INFO:tasks.workunit.client.0.trial040.stderr:Error ENXIO: problem getting command descriptions from osd.0 2026-02-13T07:01:49.493 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-13T07:01:49.493 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-13T07:01:49.493 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-13T07:01:49.493 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:01:49.493 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-13T07:01:49.493 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 0 2026-02-13T07:01:49.494 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:01:49.494 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:01:49.494 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:01:49.494 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:01:49.495 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:01:49.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:01:49.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:01:49.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 0 >= 31 )) 2026-02-13T07:01:49.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 1 2026-02-13T07:01:50.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:01:50.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:01:50.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 1' 2026-02-13T07:01:50.545 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 1 2026-02-13T07:01:50.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:01:50.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:01:50.546 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:01:50.546 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:01:50.546 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:01:50.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:01:50.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:01:50.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 1 >= 31 )) 2026-02-13T07:01:50.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 2 2026-02-13T07:01:52.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:01:52.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:01:52.600 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 2 2026-02-13T07:01:52.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 2' 2026-02-13T07:01:52.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:01:52.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:01:52.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:01:52.600 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:01:52.600 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:01:52.653 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:01:52.653 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:01:52.653 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 2 >= 31 )) 2026-02-13T07:01:52.653 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:01:55.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:01:55.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:01:55.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 3' 2026-02-13T07:01:55.655 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 3 2026-02-13T07:01:55.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:01:55.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:01:55.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:01:55.656 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:01:55.656 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:01:55.706 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:01:55.706 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:01:55.706 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 3 >= 31 )) 2026-02-13T07:01:55.706 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:01:58.707 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:01:58.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:01:58.708 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 4' 2026-02-13T07:01:58.708 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 4 2026-02-13T07:01:58.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:01:58.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:01:58.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:01:58.710 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:01:58.710 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:01:58.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:01:58.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:01:58.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 4 >= 31 )) 2026-02-13T07:01:58.760 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:01.761 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:01.762 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:01.762 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 5' 2026-02-13T07:02:01.762 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 5 2026-02-13T07:02:01.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:01.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:01.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:01.763 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:01.763 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:01.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:01.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:01.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 5 >= 31 )) 2026-02-13T07:02:01.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:04.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:04.816 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:04.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 6' 2026-02-13T07:02:04.817 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 6 2026-02-13T07:02:04.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:04.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:04.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:04.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:04.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:04.869 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:04.869 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:04.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 6 >= 31 )) 2026-02-13T07:02:04.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:07.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:07.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:07.870 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 7 2026-02-13T07:02:07.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 7' 2026-02-13T07:02:07.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:07.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:07.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:07.872 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:07.872 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:07.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:07.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:07.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 7 >= 31 )) 2026-02-13T07:02:07.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:10.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:10.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:10.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 8' 2026-02-13T07:02:10.926 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 8 2026-02-13T07:02:10.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:10.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:10.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:10.927 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:10.927 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:10.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:10.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:10.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 8 >= 31 )) 2026-02-13T07:02:10.979 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:13.980 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:13.980 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:13.980 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 9' 2026-02-13T07:02:13.980 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 9 2026-02-13T07:02:13.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:13.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:13.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:13.982 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:13.982 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:14.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:14.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:14.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 9 >= 31 )) 2026-02-13T07:02:14.033 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:17.034 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 10' 2026-02-13T07:02:17.034 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 10 2026-02-13T07:02:17.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:17.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:17.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:17.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:17.036 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:17.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:17.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:17.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 10 >= 31 )) 2026-02-13T07:02:17.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:20.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:20.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:20.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 11' 2026-02-13T07:02:20.087 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 11 2026-02-13T07:02:20.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:20.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:20.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:20.087 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:20.087 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:20.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:20.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:20.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 11 >= 31 )) 2026-02-13T07:02:20.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:23.140 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:23.140 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:23.140 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 12' 2026-02-13T07:02:23.140 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 12 2026-02-13T07:02:23.140 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:23.141 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:23.141 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:23.141 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:23.141 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:23.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:23.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:23.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 12 >= 31 )) 2026-02-13T07:02:23.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:26.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:26.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:26.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 13' 2026-02-13T07:02:26.194 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 13 2026-02-13T07:02:26.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:26.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:26.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:26.195 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:26.195 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:26.246 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:26.246 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:26.246 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 13 >= 31 )) 2026-02-13T07:02:26.246 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:29.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:29.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:29.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 14' 2026-02-13T07:02:29.249 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 14 2026-02-13T07:02:29.249 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:29.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:29.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:29.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:29.250 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:29.301 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:29.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:29.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 14 >= 31 )) 2026-02-13T07:02:29.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:32.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:32.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:32.302 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 15' 2026-02-13T07:02:32.303 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 15 2026-02-13T07:02:32.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:32.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:32.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:32.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:32.303 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:32.355 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:32.355 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:32.356 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 15 >= 31 )) 2026-02-13T07:02:32.356 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:35.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:35.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:35.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 16' 2026-02-13T07:02:35.357 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 16 2026-02-13T07:02:35.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:35.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:35.357 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:35.357 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:35.358 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:35.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:35.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:35.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 16 >= 31 )) 2026-02-13T07:02:35.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:38.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:38.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:38.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 17' 2026-02-13T07:02:38.411 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 17 2026-02-13T07:02:38.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:38.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:38.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:38.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:38.412 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:38.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:38.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:38.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 17 >= 31 )) 2026-02-13T07:02:38.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:41.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:41.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:41.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 18' 2026-02-13T07:02:41.465 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 18 2026-02-13T07:02:41.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:41.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:41.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:41.466 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:41.466 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:41.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:41.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:41.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 18 >= 31 )) 2026-02-13T07:02:41.517 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:44.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:44.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:44.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 19' 2026-02-13T07:02:44.518 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 19 2026-02-13T07:02:44.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:44.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:44.518 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:44.519 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:44.519 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:44.570 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:44.570 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:44.570 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 19 >= 31 )) 2026-02-13T07:02:44.570 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:47.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:47.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:47.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 20' 2026-02-13T07:02:47.572 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 20 2026-02-13T07:02:47.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:47.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:47.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:47.574 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:47.574 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:47.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:47.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:47.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 20 >= 31 )) 2026-02-13T07:02:47.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:50.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:50.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:50.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 21' 2026-02-13T07:02:50.627 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 21 2026-02-13T07:02:50.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:50.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:50.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:50.628 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:50.628 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:50.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:50.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:50.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 21 >= 31 )) 2026-02-13T07:02:50.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:53.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:53.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:53.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 22' 2026-02-13T07:02:53.680 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 22 2026-02-13T07:02:53.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:53.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:53.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:53.682 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:53.682 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:53.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:53.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:53.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 22 >= 31 )) 2026-02-13T07:02:53.735 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:56.736 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:56.736 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:56.736 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 23' 2026-02-13T07:02:56.736 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 23 2026-02-13T07:02:56.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:56.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:56.737 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:56.737 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:56.738 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:56.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:56.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:56.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 23 >= 31 )) 2026-02-13T07:02:56.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:02:59.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:02:59.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:02:59.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 24' 2026-02-13T07:02:59.792 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 24 2026-02-13T07:02:59.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:02:59.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:02:59.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:02:59.793 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:02:59.793 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:02:59.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:02:59.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:02:59.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 24 >= 31 )) 2026-02-13T07:02:59.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:03:02.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:03:02.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:03:02.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 25' 2026-02-13T07:03:02.847 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 25 2026-02-13T07:03:02.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:03:02.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:03:02.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:03:02.848 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:03:02.848 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:03:02.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:03:02.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:03:02.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 25 >= 31 )) 2026-02-13T07:03:02.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:03:05.903 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:03:05.903 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:03:05.903 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 26' 2026-02-13T07:03:05.904 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 26 2026-02-13T07:03:05.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:03:05.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:03:05.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:03:05.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:03:05.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:03:05.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:03:05.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:03:05.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 26 >= 31 )) 2026-02-13T07:03:05.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:03:08.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:03:08.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:03:08.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 27' 2026-02-13T07:03:08.955 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 27 2026-02-13T07:03:08.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:03:08.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:03:08.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:03:08.956 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:03:08.956 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:03:09.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:03:09.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:03:09.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 27 >= 31 )) 2026-02-13T07:03:09.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:03:12.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:03:12.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:03:12.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 28' 2026-02-13T07:03:12.011 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 28 2026-02-13T07:03:12.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:03:12.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:03:12.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:03:12.011 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:03:12.011 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:03:12.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:03:12.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:03:12.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 28 >= 31 )) 2026-02-13T07:03:12.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:03:15.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:03:15.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:03:15.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 29' 2026-02-13T07:03:15.066 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 29 2026-02-13T07:03:15.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:03:15.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:03:15.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:03:15.067 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:03:15.067 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:03:15.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:03:15.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:03:15.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 29 >= 31 )) 2026-02-13T07:03:15.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:03:18.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:03:18.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:03:18.119 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 30 2026-02-13T07:03:18.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 30' 2026-02-13T07:03:18.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:03:18.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:03:18.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:03:18.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:03:18.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:03:18.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:03:18.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:03:18.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 30 >= 31 )) 2026-02-13T07:03:18.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 3 2026-02-13T07:03:21.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T07:03:21.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T07:03:21.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 31' 2026-02-13T07:03:21.171 INFO:tasks.workunit.client.0.trial040.stdout:#---------- 1.0 loop 31 2026-02-13T07:03:21.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T07:03:21.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T07:03:21.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T07:03:21.172 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T07:03:21.172 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T07:03:21.223 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T07:03:21.223 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T07:03:21.223 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 31 >= 31 )) 2026-02-13T07:03:21.223 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: ceph report 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_fingerprint": "01acd049-04a3-4739-8edb-4b52e4d29fd1", 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "version": "20.2.0-636-g3b40640f", 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "commit": "3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb", 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "timestamp": "2026-02-13T07:03:21.525995+0000", 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "tag": "", 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "health": { 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "status": "HEALTH_WARN", 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "checks": { 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "OSD_DOWN": { 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "message": "1 osds down", 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "count": 1 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [ 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "message": "osd.0 (root=default,host=trial040) is down" 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "OSD_HOST_DOWN": { 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:03:21.528 INFO:tasks.workunit.client.0.trial040.stdout: "message": "1 host (1 osds) down", 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: "count": 1 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [ 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: "message": "host trial040 (root=default) (1 osds) is down" 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: "OSD_ROOT_DOWN": { 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:03:21.529 INFO:tasks.workunit.client.0.trial040.stdout: "message": "1 root (1 osds) down", 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: "count": 1 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [ 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: "message": "root default (1 osds) is down" 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: "PG_AVAILABILITY": { 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:03:21.530 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: "message": "Reduced data availability: 4 pgs stale", 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: "count": 4 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [ 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: "message": "pg 1.0 is stuck stale for 91s, current state stale+active+clean, last acting [0]" 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: "message": "pg 1.1 is stuck stale for 91s, current state stale+active+clean, last acting [0]" 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.531 INFO:tasks.workunit.client.0.trial040.stdout: "message": "pg 1.2 is stuck stale for 91s, current state stale+active+clean, last acting [0]" 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: "message": "pg 1.3 is stuck stale for 91s, current state stale+active+clean, last acting [0]" 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: "POOL_NO_REDUNDANCY": { 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: "message": "1 pool(s) have no replicas configured", 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: "count": 1 2026-02-13T07:03:21.532 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [ 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: "message": "pool 'rbd' has no replicas configured" 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: "TOO_FEW_OSDS": { 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: "message": "OSD count 1 < osd_pool_default_size 3", 2026-02-13T07:03:21.533 INFO:tasks.workunit.client.0.trial040.stdout: "count": 2 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [], 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "mutes": [] 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "monmap_first_committed": 1, 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "monmap_last_committed": 1, 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "monmap": { 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 1, 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "fsid": "1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2", 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "modified": "2026-02-13T06:46:44.416457Z", 2026-02-13T07:03:21.534 INFO:tasks.workunit.client.0.trial040.stdout: "created": "2026-02-13T06:46:44.416457Z", 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "min_mon_release": 20, 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "min_mon_release_name": "tentacle", 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "election_strategy": 1, 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "disallowed_leaders": "", 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode": false, 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "tiebreaker_mon": "", 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "removed_ranks": "", 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "features": { 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "persistent": [ 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "kraken", 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "luminous", 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "mimic", 2026-02-13T07:03:21.535 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap-prune", 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "nautilus", 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "octopus", 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "pacific", 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "elector-pinging", 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "quincy", 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "reef", 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "squid", 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "tentacle", 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "nvmeof_beacon_diff" 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "optional": [] 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.536 INFO:tasks.workunit.client.0.trial040.stdout: "mons": [ 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: "rank": 0, 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: "name": "a", 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: "public_addrs": { 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:7109", 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 0 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.537 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:7109/0", 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: "public_addr": "127.0.0.1:7109/0", 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: "priority": 0, 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 0, 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: "crush_location": "{}" 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: "quorum": [ 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: 0 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap": { 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 13, 2026-02-13T07:03:21.538 INFO:tasks.workunit.client.0.trial040.stdout: "fsid": "1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2", 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "created": "2026-02-13T06:46:44.453477+0000", 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "modified": "2026-02-13T07:01:49.482461+0000", 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "last_up_change": "2026-02-13T07:01:49.482461+0000", 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "last_in_change": "2026-02-13T06:46:44.765840+0000", 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "flags_num": 5799936, 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "flags_set": [ 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "pglog_hardlimit", 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "purged_snapdirs", 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "recovery_deletes", 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "sortbitwise" 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.539 INFO:tasks.workunit.client.0.trial040.stdout: "crush_version": 3, 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "full_ratio": 0.99000000953674316, 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_snapshot": "", 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "pool_max": 1, 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "max_osd": 1, 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "require_min_compat_client": "luminous", 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "min_compat_client": "jewel", 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "require_osd_release": "tentacle", 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "allow_crimson": false, 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: "pools": [ 2026-02-13T07:03:21.540 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "pool_name": "rbd", 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "create_time": "2026-02-13T06:46:46.927397+0000", 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "flags": 8193, 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "type": 1, 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "size": 1, 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "min_size": 1, 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "crush_rule": 0, 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_count": 0, 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_target": 0, 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_barrier": 0, 2026-02-13T07:03:21.541 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "is_stretch_pool": false, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "object_hash": 2, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "pg_autoscale_mode": "off", 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "pg_num": 4, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "pg_placement_num": 4, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "pg_placement_num_target": 4, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "pg_num_target": 4, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "pg_num_pending": 4, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "last_pg_merge_meta": { 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "source_pgid": "0.0", 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "ready_epoch": 0, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "last_epoch_started": 0, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "last_epoch_clean": 0, 2026-02-13T07:03:21.542 INFO:tasks.workunit.client.0.trial040.stdout: "source_version": "0'0", 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "target_version": "0'0" 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "last_change": "11", 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "last_force_op_resend": "0", 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "last_force_op_resend_preluminous": "0", 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "auid": 0, 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "snap_mode": "selfmanaged", 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "snap_seq": 2, 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "snap_epoch": 11, 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "pool_snaps": [], 2026-02-13T07:03:21.543 INFO:tasks.workunit.client.0.trial040.stdout: "removed_snaps": "[]", 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "quota_max_bytes": 0, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "quota_max_objects": 0, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "tiers": [], 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "tier_of": -1, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "read_tier": -1, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "write_tier": -1, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "cache_mode": "none", 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "target_max_bytes": 0, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "target_max_objects": 0, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-13T07:03:21.544 INFO:tasks.workunit.client.0.trial040.stdout: "cache_min_flush_age": 0, 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "cache_min_evict_age": 0, 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "erasure_code_profile": "", 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_params": { 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "type": "none" 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_period": 0, 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_count": 0, 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "use_gmt_hitset": true, 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "min_read_recency_for_promote": 0, 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "min_write_recency_for_promote": 0, 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_grade_decay_rate": 0, 2026-02-13T07:03:21.545 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_search_last_n": 0, 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "grade_table": [], 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "stripe_width": 0, 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "expected_num_objects": 0, 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "fast_read": false, 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "nonprimary_shards": "{}", 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "options": {}, 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "application_metadata": { 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "rbd": {} 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "osds": [ 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.546 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "uuid": "15e2566a-9217-4e72-b873-28f8820db63c", 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "up": 0, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "in": 1, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 1, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "primary_affinity": 1, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "last_clean_begin": 0, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "last_clean_end": 0, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "up_from": 5, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "up_thru": 6, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "down_at": 13, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "lost_at": 0, 2026-02-13T07:03:21.547 INFO:tasks.workunit.client.0.trial040.stdout: "public_addrs": { 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6800", 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1782860686 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6801", 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1782860686 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.548 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_addrs": { 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6802", 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1782860686 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6803", 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1782860686 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.549 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_back_addrs": { 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6806", 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1782860686 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6807", 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1782860686 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.550 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_front_addrs": { 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6804", 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1782860686 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6805", 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1782860686 2026-02-13T07:03:21.551 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: "public_addr": "127.0.0.1:6801/1782860686", 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_addr": "127.0.0.1:6803/1782860686", 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_back_addr": "127.0.0.1:6807/1782860686", 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_front_addr": "127.0.0.1:6805/1782860686", 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: "state": [ 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: "exists" 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: "osd_xinfo": [ 2026-02-13T07:03:21.552 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "down_stamp": "2026-02-13T07:01:49.482461+0000", 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "laggy_probability": 0, 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "laggy_interval": 0, 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "features": 4544132024016699391, 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "old_weight": 0, 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "last_purged_snaps_scrub": "2026-02-13T06:46:47.024855+0000", 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "dead_epoch": 0 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "pg_upmap": [], 2026-02-13T07:03:21.553 INFO:tasks.workunit.client.0.trial040.stdout: "pg_upmap_items": [], 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "pg_upmap_primaries": [], 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "pg_temp": [], 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "primary_temp": [], 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "blocklist": {}, 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "range_blocklist": {}, 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "erasure_code_profiles": { 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "default": { 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "k": "2", 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "m": "2", 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "plugin": "isa", 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "technique": "reed_sol_van" 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.554 INFO:tasks.workunit.client.0.trial040.stdout: "removed_snaps_queue": [], 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "new_removed_snaps": [], 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "new_purged_snaps": [], 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "crush_node_flags": {}, 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "device_class_flags": {}, 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode": { 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode_enabled": false, 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_bucket_count": 0, 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "degraded_stretch_mode": 0, 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "recovering_stretch_mode": 0, 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode_bucket": 0 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.555 INFO:tasks.workunit.client.0.trial040.stdout: "osd_metadata": [ 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "arch": "x86_64", 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "back_addr": "[v2:127.0.0.1:6802/1782860686,v1:127.0.0.1:6803/1782860686]", 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "back_iface": "", 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs": "1", 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs_dedicated_db": "0", 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs_dedicated_wal": "0", 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs_single_shared_device": "1", 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_allocation_from_file": "1", 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_allocator": "hybrid", 2026-02-13T07:03:21.556 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_access_mode": "file", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_block_size": "4096", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_path": "td/ceph-helpers/0/block", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_rotational": "0", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_size": "107374182400", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_support_discard": "1", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_type": "ssd", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_min_alloc_size": "4096", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_write_mode": "classic", 2026-02-13T07:03:21.557 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_release": "tentacle", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_version": "ceph version 20.2.0-636-g3b40640f (3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb) tentacle (stable - RelWithDebInfo)", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_version_short": "20.2.0-636-g3b40640f", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_version_when_created": "ceph version 20.2.0-636-g3b40640f (3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb) tentacle (stable - RelWithDebInfo)", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "created_at": "2026-02-13T06:46:45.154413Z", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "default_device_class": "ssd", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6793", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "devices": "nvme0n1", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "distro": "centos", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "distro_description": "CentOS Stream 9", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "distro_version": "9", 2026-02-13T07:03:21.558 INFO:tasks.workunit.client.0.trial040.stdout: "front_addr": "[v2:127.0.0.1:6800/1782860686,v1:127.0.0.1:6801/1782860686]", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "front_iface": "", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/1782860686,v1:127.0.0.1:6807/1782860686]", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/1782860686,v1:127.0.0.1:6805/1782860686]", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "hostname": "trial040", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "journal_rotational": "0", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 6 13:57:07 UTC 2026", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "kernel_version": "5.14.0-677.el9.x86_64", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "mem_swap_kb": "8388604", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "mem_total_kb": "130438944", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "objectstore_numa_node": "0", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "objectstore_numa_nodes": "0", 2026-02-13T07:03:21.559 INFO:tasks.workunit.client.0.trial040.stdout: "os": "Linux", 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "osd_data": "td/ceph-helpers/0", 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "osd_objectstore": "bluestore", 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "osdspec_affinity": "", 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "rotational": "0" 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap_clean_epochs": { 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "min_last_epoch_clean": 0, 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "last_epoch_clean": { 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "per_pool": [] 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "osd_epochs": [] 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.560 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap_first_committed": 1, 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap_last_committed": 13, 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: "crushmap": { 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: "devices": [ 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: "name": "osd.0", 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: "class": "ssd" 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: "types": [ 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 0, 2026-02-13T07:03:21.561 INFO:tasks.workunit.client.0.trial040.stdout: "name": "osd" 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 1, 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: "name": "host" 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 2, 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: "name": "chassis" 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 3, 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: "name": "rack" 2026-02-13T07:03:21.562 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 4, 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: "name": "row" 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 5, 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: "name": "pdu" 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 6, 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: "name": "pod" 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.563 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 7, 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: "name": "room" 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 8, 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: "name": "datacenter" 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 9, 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: "name": "zone" 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 10, 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: "name": "region" 2026-02-13T07:03:21.564 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 11, 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "name": "root" 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "buckets": [ 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "id": -1, 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "name": "default", 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 11, 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "root", 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:03:21.565 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "id": -3, 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "id": -2, 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "name": "default~ssd", 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 11, 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "root", 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:03:21.566 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "id": -4, 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "id": -3, 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "name": "trial040", 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 1, 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "host", 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:03:21.567 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "id": -4, 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "name": "trial040~ssd", 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 1, 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "host", 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:03:21.568 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "rules": [ 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "rule_id": 0, 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "rule_name": "replicated_rule", 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "type": 1, 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "steps": [ 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "op": "take", 2026-02-13T07:03:21.569 INFO:tasks.workunit.client.0.trial040.stdout: "item": -1, 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: "item_name": "default" 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: "op": "choose_firstn", 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: "num": 0, 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: "type": "osd" 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: "op": "emit" 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: "tunables": { 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: "choose_local_tries": 0, 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: "choose_local_fallback_tries": 0, 2026-02-13T07:03:21.570 INFO:tasks.workunit.client.0.trial040.stdout: "choose_total_tries": 50, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "chooseleaf_descend_once": 1, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "chooseleaf_vary_r": 1, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "chooseleaf_stable": 1, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "msr_descents": 100, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "msr_collision_tries": 100, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "straw_calc_version": 1, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "allowed_bucket_algs": 54, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "profile": "jewel", 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "optimal_tunables": 1, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "legacy_tunables": 0, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "minimum_required_version": "jewel", 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables": 1, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables2": 1, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "has_v2_rules": 0, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables3": 1, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "has_v3_rules": 0, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "has_v4_buckets": 1, 2026-02-13T07:03:21.571 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables5": 1, 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "has_v5_rules": 0, 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "has_msr_rules": 0 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "choose_args": {} 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "fsmap": { 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 1, 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "btime": "2026-02-13T06:46:44:453312+0000", 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "default_fscid": -1, 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "compat": { 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "compat": {}, 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "ro_compat": {}, 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "incompat": { 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "feature_1": "base v0.20", 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "feature_2": "client writeable ranges", 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "feature_3": "default file layouts on dirs", 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "feature_4": "dir inode in separate object", 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "feature_5": "mds uses versioned encoding", 2026-02-13T07:03:21.572 INFO:tasks.workunit.client.0.trial040.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "feature_8": "no anchor table", 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "feature_9": "file layout v2", 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "feature_10": "snaprealm v2", 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "feature_11": "minor log segments", 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "feature_12": "quiesce subvolumes" 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "feature_flags": { 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "enable_multiple": true, 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "ever_enabled_multiple": true 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "standbys": [], 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "filesystems": [] 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "mdsmap_first_committed": 1, 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "mdsmap_last_committed": 1, 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "auth": { 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "first_committed": 1, 2026-02-13T07:03:21.573 INFO:tasks.workunit.client.0.trial040.stdout: "last_committed": 3, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_secrets": 1 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg": 4, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_active": 4, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_unknown": 0, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_osd": 1, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "pool_sum": { 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "stat_sum": { 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes": 19, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects": 2, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_clones": 0, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_copies": 2, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing": 0, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_degraded": 0, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_misplaced": 0, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_unfound": 0, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_dirty": 2, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_whiteouts": 0, 2026-02-13T07:03:21.574 INFO:tasks.workunit.client.0.trial040.stdout: "num_read": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_read_kb": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_write": 2, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_write_kb": 2, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_scrub_errors": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_deep_scrub_errors": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_recovered": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_recovered": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_keys_recovered": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_omap": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_kb": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_kb": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_promote": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_high": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_low": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_some": 0, 2026-02-13T07:03:21.575 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_full": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_pinned": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "num_legacy_snapsets": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "num_large_omap_objects": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_manifest": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_bytes": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_keys": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_repaired": 0 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "store_stats": { 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "total": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "available": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 0, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 0 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.576 INFO:tasks.workunit.client.0.trial040.stdout: "log_size": 3, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "ondisk_log_size": 3, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "up": 4, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "acting": 4, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "num_store_stats": 0 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "osd_sum": { 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "up_from": 0, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "seq": 0, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "num_pgs": 4, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "num_osds": 1, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_osds": 1, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_omap_osds": 1, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "kb": 104857600, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used": 27068, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_data": 108, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_omap": 8, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_meta": 26935, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "kb_avail": 104830532, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "statfs": { 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "total": 107374182400, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "available": 107346464768, 2026-02-13T07:03:21.577 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 110592, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 23298, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 8863, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 27581793 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "hb_peers": [], 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "snap_trim_queue_len": 0, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "num_snap_trimming": 0, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "num_shards_repaired": 0, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "op_queue_age_hist": { 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "histogram": [], 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "upper_bound": 1 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "perf_stat": { 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ms": 0, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ms": 0, 2026-02-13T07:03:21.578 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ns": 0, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ns": 0 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "alerts": [], 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "network_ping_times": [] 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "osd_sum_by_class": { 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "ssd": { 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "up_from": 0, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "seq": 0, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "num_pgs": 4, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "num_osds": 1, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_osds": 1, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_omap_osds": 1, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "kb": 104857600, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used": 27068, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_data": 108, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_omap": 8, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_meta": 26935, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "kb_avail": 104830532, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "statfs": { 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "total": 107374182400, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "available": 107346464768, 2026-02-13T07:03:21.579 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 110592, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 23298, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 8863, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 27581793 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "hb_peers": [], 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "snap_trim_queue_len": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "num_snap_trimming": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "num_shards_repaired": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "op_queue_age_hist": { 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "histogram": [], 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "upper_bound": 1 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "perf_stat": { 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ms": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ms": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ns": 0, 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ns": 0 2026-02-13T07:03:21.580 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "alerts": [], 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "network_ping_times": [] 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "pool_stats": [ 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "poolid": 1, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg": 4, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "stat_sum": { 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes": 19, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects": 2, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_clones": 0, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_copies": 2, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing": 0, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_degraded": 0, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_misplaced": 0, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_unfound": 0, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_dirty": 2, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_whiteouts": 0, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_read": 0, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_read_kb": 0, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_write": 2, 2026-02-13T07:03:21.581 INFO:tasks.workunit.client.0.trial040.stdout: "num_write_kb": 2, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_scrub_errors": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_deep_scrub_errors": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_recovered": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_recovered": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_keys_recovered": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_omap": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_kb": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_kb": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_promote": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_high": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_low": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_some": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_full": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_pinned": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_legacy_snapsets": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_large_omap_objects": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_manifest": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_bytes": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_keys": 0, 2026-02-13T07:03:21.582 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_repaired": 0 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "store_stats": { 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "total": 0, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "available": 0, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 4096, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 19, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 0, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 0 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "log_size": 3, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "ondisk_log_size": 3, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "up": 4, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "acting": 4, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "num_store_stats": 1 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "osd_stats": [ 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:03:21.583 INFO:tasks.workunit.client.0.trial040.stdout: "seq": 21474836487 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_by_state": [ 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "state": "stale+active+clean", 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "num": 4 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "pool_pg_unavailable_map": [ 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "poolid": "1", 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "pgs": [ 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "1.3", 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "1.2", 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "1.1", 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "1.0" 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_by_osd": [ 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "num_primary_pg": 4, 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "num_acting_pg": 4, 2026-02-13T07:03:21.584 INFO:tasks.workunit.client.0.trial040.stdout: "num_up_not_acting_pg": 0 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "purged_snaps": [ 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "purged_snaps": {} 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "servicemap": { 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 3, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "modified": "2026-02-13T07:02:48.764922+0000", 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "services": {} 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "mgrstat_first_committed": 253, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "mgrstat_last_committed": 499, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "logm_first_committed": 1, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "logm_last_committed": 544, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "paxos": { 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "first_committed": 503, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "last_committed": 1068, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "last_pn": 0, 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: "accepted_pn": 0 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:03:21.585 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T07:03:21.586 INFO:tasks.workunit.client.0.trial040.stderr:report 2698818781 2026-02-13T07:03:21.586 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: echo 'PG 1.0 is not clean after 31 iterations' 2026-02-13T07:03:21.586 INFO:tasks.workunit.client.0.trial040.stdout:PG 1.0 is not clean after 31 iterations 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1714: wait_for_pg_clean: return 1 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: return 1 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1952: test_pg_scrub: teardown td/ceph-helpers 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:21.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:21.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:21.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:03:21.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:03:21.645 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:03:21.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:03:21.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:03:21.646 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:03:21.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:21.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:03:21.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:03:21.648 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:21.649 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:03:21.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:03:21.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:03:21.660 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:03:21.660 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:21.660 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:21.660 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:03:21.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:03:21.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:03:21.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:03:21.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_repair td/ceph-helpers 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1894: test_repair: local dir=td/ceph-helpers 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: test_repair: setup td/ceph-helpers 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:21.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:21.664 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:03:21.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:03:21.665 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:03:21.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:03:21.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:03:21.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:03:21.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:21.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:03:21.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:03:21.667 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:21.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:03:21.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:03:21.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:03:21.670 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:03:21.670 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:21.670 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:21.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:03:21.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:03:21.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:03:21.671 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:03:21.672 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:03:21.672 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:21.672 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:21.672 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:03:21.674 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:03:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:03:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:03:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:03:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: test_repair: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:03:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:03:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:03:21.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:03:21.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:03:21.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:03:21.675 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:03:21.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:03:21.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:21.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:21.697 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:21.697 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:21.697 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:21.697 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:21.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:03:21.725 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:03:21.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:03:21.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:21.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:21.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:21.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:03:21.770 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:03:21.770 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:21.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:21.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:03:21.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:03:21.772 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:21.772 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:21.772 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:21.772 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:21.772 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:21.772 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:21.773 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:21.773 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:21.773 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:03:21.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: test_repair: run_mgr td/ceph-helpers x 2026-02-13T07:03:21.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T07:03:21.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:03:21.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:03:21.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:03:21.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T07:03:21.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:03:21.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:03:21.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:21.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:21.918 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:21.918 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:21.919 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:21.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:21.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:03:21.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:03:21.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: test_repair: run_osd td/ceph-helpers 0 2026-02-13T07:03:21.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:03:21.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:03:21.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=343b912f-1ae0-4087-abe6-e26e9e8b4cb2 2026-02-13T07:03:21.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 343b912f-1ae0-4087-abe6-e26e9e8b4cb2' 2026-02-13T07:03:21.943 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 343b912f-1ae0-4087-abe6-e26e9e8b4cb2 2026-02-13T07:03:21.943 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:03:21.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC5zI5pt6uTOBAAaL6rPHK2FtycvqDwgimEGw== 2026-02-13T07:03:21.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC5zI5pt6uTOBAAaL6rPHK2FtycvqDwgimEGw=="}' 2026-02-13T07:03:21.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 343b912f-1ae0-4087-abe6-e26e9e8b4cb2 -i td/ceph-helpers/0/new.json 2026-02-13T07:03:22.057 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:03:22.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T07:03:22.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC5zI5pt6uTOBAAaL6rPHK2FtycvqDwgimEGw== --osd-uuid 343b912f-1ae0-4087-abe6-e26e9e8b4cb2 2026-02-13T07:03:22.084 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:22.081+0000 7f6a05472900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:22.086 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:22.083+0000 7f6a05472900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:22.087 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:22.084+0000 7f6a05472900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:22.087 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:22.084+0000 7f6a05472900 -1 bdev(0x55e84f698c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:03:22.087 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:22.084+0000 7f6a05472900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T07:03:22.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T07:03:22.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:03:22.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:03:22.556 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:03:22.556 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:03:22.663 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:03:22.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:03:22.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:03:22.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:03:22.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:03:22.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:03:22.680 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:22.677+0000 7fe0a6e15900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:22.682 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:22.679+0000 7fe0a6e15900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:22.683 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:22.680+0000 7fe0a6e15900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:22.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:03:22.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:03:22.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:03:22.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:03:22.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:03:22.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:22.764 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:03:22.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:03:22.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:22.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:03:22.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:03:22.961 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:22.958+0000 7fe0a6e15900 -1 Falling back to public interface 2026-02-13T07:03:23.424 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:23.421+0000 7fe0a6e15900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:03:23.866 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:03:23.866 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:23.866 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:03:23.866 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:03:23.866 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:23.866 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:03:23.970 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:03:24.971 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:03:24.971 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:24.971 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:03:24.971 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-13T07:03:24.971 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:24.971 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:03:25.199 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3665557390,v1:127.0.0.1:6801/3665557390] [v2:127.0.0.1:6802/3665557390,v1:127.0.0.1:6803/3665557390] exists,up 343b912f-1ae0-4087-abe6-e26e9e8b4cb2 2026-02-13T07:03:25.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:03:25.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:03:25.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:03:25.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1900: test_repair: create_rbd_pool 2026-02-13T07:03:25.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T07:03:25.421 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T07:03:25.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T07:03:25.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T07:03:25.706 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T07:03:25.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T07:03:26.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T07:03:26.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1901: test_repair: wait_for_clean 2026-02-13T07:03:26.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:03:26.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:03:26.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:03:26.997 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T07:03:26.997 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:03:26.997 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:03:26.998 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:03:26.998 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:03:26.998 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:03:27.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:03:27.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:03:27.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:03:27.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:03:27.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:03:27.079 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:03:27.317 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T07:03:27.317 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:03:27.317 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:03:27.318 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:03:27.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T07:03:27.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T07:03:27.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T07:03:27.381 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:03:27.382 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T07:03:27.382 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:03:27.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:03:27.383 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T07:03:27.383 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:03:27.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T07:03:27.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T07:03:27.385 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T07:03:27.385 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:03:27.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T07:03:27.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T07:03:28.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T07:03:28.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:03:28.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T07:03:28.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:03:28.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:03:28.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:03:29.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T07:03:29.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:03:29.163 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:03:29.164 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:03:29.164 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:03:29.164 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:03:29.164 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:03:29.164 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:03:29.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T07:03:29.385 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:03:29.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:03:29.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:03:29.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T07:03:29.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T07:03:29.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T07:03:29.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1902: test_repair: repair 1.0 2026-02-13T07:03:29.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.0 2026-02-13T07:03:29.697 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.0 2026-02-13T07:03:29.698 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:29.698 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:29.698 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:29.698 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:29.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-13T07:03:25.642476+0000 2026-02-13T07:03:29.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.0 2026-02-13T07:03:30.131 INFO:tasks.workunit.client.0.trial040.stderr:instructing pg 1.0 on osd.0 to repair 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.0 2026-02-13T07:03:25.642476+0000 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-13T07:03:25.642476+0000 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:30.139 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:30.368 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:03:25.642476+0000 '>' 2026-02-13T07:03:25.642476+0000 2026-02-13T07:03:30.368 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:03:31.369 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:03:31.369 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T07:03:31.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:03:31.371 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:31.371 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:31.371 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:31.371 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:31.598 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:03:25.642476+0000 '>' 2026-02-13T07:03:25.642476+0000 2026-02-13T07:03:31.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:03:32.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:03:32.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T07:03:32.600 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:03:32.600 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:32.600 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:32.601 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:32.601 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:32.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:03:25.642476+0000 '>' 2026-02-13T07:03:25.642476+0000 2026-02-13T07:03:32.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:03:33.822 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:03:33.822 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T07:03:33.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:03:33.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:33.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:33.823 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:33.824 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:34.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:03:25.642476+0000 '>' 2026-02-13T07:03:25.642476+0000 2026-02-13T07:03:34.047 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:03:35.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:03:35.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T07:03:35.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:03:35.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:35.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:35.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:35.049 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:35.273 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:03:25.642476+0000 '>' 2026-02-13T07:03:25.642476+0000 2026-02-13T07:03:35.274 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:03:36.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:03:36.275 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T07:03:36.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:03:36.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:36.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:36.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:36.276 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:36.500 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:03:30.414047+0000 '>' 2026-02-13T07:03:25.642476+0000 2026-02-13T07:03:36.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-13T07:03:36.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1903: test_repair: kill_daemons td/ceph-helpers KILL osd 2026-02-13T07:03:36.501 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:36.501 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:36.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:36.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:36.501 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:36.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:36.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1904: test_repair: TIMEOUT=2 2026-02-13T07:03:36.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1904: test_repair: repair 1.0 2026-02-13T07:03:36.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.0 2026-02-13T07:03:36.607 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.0 2026-02-13T07:03:36.607 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:36.607 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:36.608 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:36.608 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:36.836 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-13T07:03:30.414047+0000 2026-02-13T07:03:36.836 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.0 2026-02-13T07:03:37.056 INFO:tasks.workunit.client.0.trial040.stderr:instructing pg 1.0 on osd.0 to repair 2026-02-13T07:03:37.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.0 2026-02-13T07:03:30.414047+0000 2026-02-13T07:03:37.065 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-13T07:03:37.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-13T07:03:30.414047+0000 2026-02-13T07:03:37.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-13T07:03:37.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-13T07:03:37.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 2 )) 2026-02-13T07:03:37.066 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:03:37.066 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:37.066 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:37.066 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:37.066 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:37.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:03:30.414047+0000 '>' 2026-02-13T07:03:30.414047+0000 2026-02-13T07:03:37.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:03:38.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:03:38.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 2 )) 2026-02-13T07:03:38.288 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:03:38.288 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:03:38.288 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:03:38.288 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:03:38.288 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:03:38.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:03:30.414047+0000 '>' 2026-02-13T07:03:30.414047+0000 2026-02-13T07:03:38.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:03:39.511 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:03:39.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 2 )) 2026-02-13T07:03:39.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: return 1 2026-02-13T07:03:39.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1905: test_repair: teardown td/ceph-helpers 2026-02-13T07:03:39.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:03:39.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:03:39.512 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:03:39.512 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:39.513 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:39.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:39.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:39.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:39.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:39.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:03:39.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:03:39.620 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:03:39.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:03:39.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:03:39.622 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:03:39.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:39.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:03:39.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:03:39.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:39.624 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:03:39.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:03:39.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:03:39.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:03:39.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:39.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:39.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:03:39.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:03:39.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:03:39.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:03:39.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_run_in_background td/ceph-helpers 2026-02-13T07:03:39.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2198: test_run_in_background: local pids 2026-02-13T07:03:39.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2199: test_run_in_background: run_in_background pids sleep 1 2026-02-13T07:03:39.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-13T07:03:39.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 119227"' 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 119227' 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2200: test_run_in_background: run_in_background pids sleep 1 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 119228"' 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 119228' 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sleep 1 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sleep 1 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/119232: /' 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/119233: /' 2026-02-13T07:03:39.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: test_run_in_background: echo 119227 119228 2026-02-13T07:03:39.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: test_run_in_background: wc -w 2026-02-13T07:03:39.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: test_run_in_background: test 2 = 2 2026-02-13T07:03:39.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: test_run_in_background: wait 119227 119228 2026-02-13T07:03:40.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_run_mon td/ceph-helpers 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:486: test_run_mon: local dir=td/ceph-helpers 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:488: test_run_mon: setup td/ceph-helpers 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:03:40.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:40.635 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:40.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:40.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:40.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:40.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:40.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:03:40.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:03:40.637 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:03:40.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:03:40.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:03:40.639 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:03:40.639 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:40.640 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:03:40.640 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:03:40.640 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:40.641 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:03:40.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:03:40.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:03:40.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:03:40.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:40.643 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:40.643 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:03:40.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:03:40.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:03:40.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:03:40.645 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:03:40.645 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:40.646 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:40.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:03:40.646 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:490: test_run_mon: run_mon td/ceph-helpers a 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:03:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T07:03:40.667 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:03:40.667 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:40.667 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:40.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:40.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:40.668 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:40.668 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:40.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T07:03:40.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:03:40.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:03:40.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:40.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:40.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:03:40.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:03:40.698 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:40.698 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:40.698 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:40.698 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:40.698 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:40.698 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:40.698 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:40.698 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:40.698 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:03:40.743 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:03:40.743 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:40.743 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:40.743 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:03:40.744 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:03:40.744 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:40.744 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:40.744 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:40.745 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:40.745 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:40.745 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:40.745 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:40.745 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:40.745 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:03:40.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:491: test_run_mon: ceph mon dump 2026-02-13T07:03:40.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:491: test_run_mon: grep mon.a 2026-02-13T07:03:40.886 INFO:tasks.workunit.client.0.trial040.stderr:dumped monmap epoch 1 2026-02-13T07:03:40.891 INFO:tasks.workunit.client.0.trial040.stdout:0: v2:127.0.0.1:7109/0 mon.a 2026-02-13T07:03:40.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:492: test_run_mon: kill_daemons td/ceph-helpers 2026-02-13T07:03:40.892 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:40.893 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:40.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:40.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:40.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:41.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:41.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:494: test_run_mon: run_mon td/ceph-helpers a --osd_pool_default_size=3 2026-02-13T07:03:41.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:03:41.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:03:41.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:03:41.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:03:41.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:03:41.200 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=3 2026-02-13T07:03:41.215 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:41.211+0000 7f645a212d80 -1 'td/ceph-helpers/a' already exists and is not empty: monitor may already exist 2026-02-13T07:03:41.217 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:03:41.217 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:41.217 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:41.217 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:41.217 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:41.217 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:41.217 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:41.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-13T07:03:41.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:03:41.246 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:03:41.246 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:41.246 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:41.246 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:03:41.247 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:03:41.247 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:41.247 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:41.247 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:41.248 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:41.248 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:41.248 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:41.248 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:41.248 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:41.248 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:03:41.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:03:41.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:41.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:41.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:03:41.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:03:41.292 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:41.292 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:41.292 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:41.293 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:41.293 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:41.293 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:41.293 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:41.293 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:41.293 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:03:41.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:495: test_run_mon: run_osd td/ceph-helpers 0 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:03:41.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:41.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:03:41.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=91e24fd0-2f48-487b-b78b-3bff4817acd5 2026-02-13T07:03:41.345 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 91e24fd0-2f48-487b-b78b-3bff4817acd5' 2026-02-13T07:03:41.345 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 91e24fd0-2f48-487b-b78b-3bff4817acd5 2026-02-13T07:03:41.345 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:03:41.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDNzI5pklbvFBAA0zkfH6A2WifK7P5UKk+02Q== 2026-02-13T07:03:41.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDNzI5pklbvFBAA0zkfH6A2WifK7P5UKk+02Q=="}' 2026-02-13T07:03:41.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 91e24fd0-2f48-487b-b78b-3bff4817acd5 -i td/ceph-helpers/0/new.json 2026-02-13T07:03:41.459 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:03:41.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T07:03:41.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNzI5pklbvFBAA0zkfH6A2WifK7P5UKk+02Q== --osd-uuid 91e24fd0-2f48-487b-b78b-3bff4817acd5 2026-02-13T07:03:41.482 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:41.479+0000 7fd953f2d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:41.483 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:41.480+0000 7fd953f2d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:41.484 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:41.481+0000 7fd953f2d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:41.485 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:41.482+0000 7fd953f2d900 -1 bdev(0x564122b0cc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:03:41.485 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:41.482+0000 7fd953f2d900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T07:03:41.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T07:03:41.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:03:41.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:03:41.820 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:03:41.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:03:41.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:03:41.927 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:03:41.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:03:41.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:03:41.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:03:41.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:03:41.943 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:41.940+0000 7f8d22bc1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:41.944 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:41.941+0000 7f8d22bc1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:41.945 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:41.942+0000 7f8d22bc1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:42.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:03:42.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:03:42.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:03:42.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:03:42.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:03:42.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:42.027 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:03:42.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:03:42.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:42.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:03:42.103 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:42.100+0000 7f8d22bc1900 -1 Falling back to public interface 2026-02-13T07:03:42.122 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:03:42.363 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:42.360+0000 7f8d22bc1900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:03:43.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:03:43.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:43.124 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:03:43.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:03:43.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:43.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:03:43.223 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/975892116,v1:127.0.0.1:6801/975892116] [v2:127.0.0.1:6802/975892116,v1:127.0.0.1:6803/975892116] exists,up 91e24fd0-2f48-487b-b78b-3bff4817acd5 2026-02-13T07:03:43.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:496: test_run_mon: run_osd td/ceph-helpers 1 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:03:43.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:43.228 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:03:43.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:03:43.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:03:43.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T07:03:43.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:03:43.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6f857cb0-ddd6-4eb3-aeb1-483848f4a2bb 2026-02-13T07:03:43.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6f857cb0-ddd6-4eb3-aeb1-483848f4a2bb' 2026-02-13T07:03:43.230 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 6f857cb0-ddd6-4eb3-aeb1-483848f4a2bb 2026-02-13T07:03:43.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:03:43.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDPzI5pRdwVDhAABD/kWDM7qiUtUndrlWZGZA== 2026-02-13T07:03:43.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDPzI5pRdwVDhAABD/kWDM7qiUtUndrlWZGZA=="}' 2026-02-13T07:03:43.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6f857cb0-ddd6-4eb3-aeb1-483848f4a2bb -i td/ceph-helpers/1/new.json 2026-02-13T07:03:43.342 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:03:43.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T07:03:43.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDPzI5pRdwVDhAABD/kWDM7qiUtUndrlWZGZA== --osd-uuid 6f857cb0-ddd6-4eb3-aeb1-483848f4a2bb 2026-02-13T07:03:43.366 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:43.363+0000 7f9e188e5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:43.367 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:43.364+0000 7f9e188e5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:43.368 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:43.365+0000 7f9e188e5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:43.369 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:43.366+0000 7f9e188e5900 -1 bdev(0x557a0b0c4c00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:03:43.369 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:43.366+0000 7f9e188e5900 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T07:03:43.685 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T07:03:43.685 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:03:43.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:03:43.686 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:03:43.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:03:43.788 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:03:43.789 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:03:43.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:03:43.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:03:43.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:03:43.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:03:43.805 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:43.802+0000 7fb00beb6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:43.806 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:43.803+0000 7fb00beb6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:43.807 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:43.804+0000 7fb00beb6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:43.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:03:43.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:03:43.890 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:03:43.890 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:03:43.890 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:03:43.890 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:43.890 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:03:43.890 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:03:43.891 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:43.891 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:03:43.982 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:43.979+0000 7fb00beb6900 -1 Falling back to public interface 2026-02-13T07:03:43.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:03:44.231 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:44.226+0000 7fb00beb6900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:03:44.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:03:44.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:44.991 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:03:44.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:03:44.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:44.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:03:45.088 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/1913650485,v1:127.0.0.1:6809/1913650485] [v2:127.0.0.1:6810/1913650485,v1:127.0.0.1:6811/1913650485] exists,up 6f857cb0-ddd6-4eb3-aeb1-483848f4a2bb 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:497: test_run_mon: run_osd td/ceph-helpers 2 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/2 2026-02-13T07:03:45.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/2' 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/2/journal' 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:45.093 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:03:45.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:03:45.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:03:45.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:03:45.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:03:45.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:03:45.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/2 2026-02-13T07:03:45.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:03:45.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=87b2b717-95df-47e9-99d6-3535b17103e9 2026-02-13T07:03:45.095 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 87b2b717-95df-47e9-99d6-3535b17103e9 2026-02-13T07:03:45.096 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 87b2b717-95df-47e9-99d6-3535b17103e9' 2026-02-13T07:03:45.096 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:03:45.104 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDRzI5puWMLBhAA1ja1nWseQd+jGKc8dxFMTA== 2026-02-13T07:03:45.104 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDRzI5puWMLBhAA1ja1nWseQd+jGKc8dxFMTA=="}' 2026-02-13T07:03:45.104 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 87b2b717-95df-47e9-99d6-3535b17103e9 -i td/ceph-helpers/2/new.json 2026-02-13T07:03:45.209 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:03:45.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/2/new.json 2026-02-13T07:03:45.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/2 --osd-journal=td/ceph-helpers/2/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDRzI5puWMLBhAA1ja1nWseQd+jGKc8dxFMTA== --osd-uuid 87b2b717-95df-47e9-99d6-3535b17103e9 2026-02-13T07:03:45.234 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:45.231+0000 7f72b787c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:45.235 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:45.232+0000 7f72b787c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:45.236 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:45.233+0000 7f72b787c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:45.237 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:45.234+0000 7f72b787c900 -1 bdev(0x55a9fbfdcc00 td/ceph-helpers/2/block) open stat got: (1) Operation not permitted 2026-02-13T07:03:45.237 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:45.234+0000 7f72b787c900 -1 bluestore(td/ceph-helpers/2) _read_fsid unparsable uuid 2026-02-13T07:03:45.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/2/keyring 2026-02-13T07:03:45.554 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:03:45.555 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T07:03:45.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T07:03:45.555 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:03:45.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T07:03:45.658 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:03:45.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/2 --osd-journal=td/ceph-helpers/2/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:03:45.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:03:45.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:03:45.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:03:45.674 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:45.671+0000 7f307ed3b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:45.676 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:45.673+0000 7f307ed3b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:45.677 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:45.674+0000 7f307ed3b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:45.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T07:03:45.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:03:45.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:03:45.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:03:45.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:03:45.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:45.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:03:45.756 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:03:45.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:45.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:03:45.837 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:45.833+0000 7f307ed3b900 -1 Falling back to public interface 2026-02-13T07:03:45.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:03:46.123 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:46.119+0000 7f307ed3b900 -1 osd.2 0 log_to_monitors true 2026-02-13T07:03:46.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:03:46.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:46.854 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:03:46.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:03:46.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:46.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:03:46.954 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/3964497466,v1:127.0.0.1:6817/3964497466] [v2:127.0.0.1:6818/3964497466,v1:127.0.0.1:6819/3964497466] exists,up 87b2b717-95df-47e9-99d6-3535b17103e9 2026-02-13T07:03:46.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:03:46.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:03:46.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:03:46.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:498: test_run_mon: create_rbd_pool 2026-02-13T07:03:46.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T07:03:47.051 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T07:03:47.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T07:03:47.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T07:03:47.155 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T07:03:47.161 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T07:03:48.162 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T07:03:48.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:499: test_run_mon: ceph osd dump 2026-02-13T07:03:48.445 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:499: test_run_mon: grep 'pool 1 '\''rbd'\''' 2026-02-13T07:03:48.543 INFO:tasks.workunit.client.0.trial040.stdout:pool 1 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 4 pgp_num 4 autoscale_mode off last_change 19 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.50 2026-02-13T07:03:48.544 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:501: test_run_mon: get_asok_path mon.a 2026-02-13T07:03:48.545 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:48.545 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:48.545 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:48.545 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:48.545 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:48.545 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:48.546 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:501: test_run_mon: CEPH_ARGS= 2026-02-13T07:03:48.546 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:501: test_run_mon: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get osd_pool_default_size 2026-02-13T07:03:48.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:501: test_run_mon: local 'size={"osd_pool_default_size":"3"}' 2026-02-13T07:03:48.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:502: test_run_mon: test '{"osd_pool_default_size":"3"}' = '{"osd_pool_default_size":"3"}' 2026-02-13T07:03:48.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:504: test_run_mon: CEPH_ARGS= 2026-02-13T07:03:48.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:504: test_run_mon: ceph status 2026-02-13T07:03:48.633 INFO:tasks.workunit.client.0.trial040.stderr:unable to get monitor info from DNS SRV with service name: ceph-mon 2026-02-13T07:03:48.633 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.629+0000 7f945cb4b640 -1 failed for service _ceph-mon._tcp 2026-02-13T07:03:48.633 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.629+0000 7f945cb4b640 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2026-02-13T07:03:48.633 INFO:tasks.workunit.client.0.trial040.stderr:[errno 2] RADOS object not found (error connecting to the cluster) 2026-02-13T07:03:48.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:505: test_run_mon: CEPH_ARGS= 2026-02-13T07:03:48.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:505: test_run_mon: ceph --conf td/ceph-helpers/ceph.conf status 2026-02-13T07:03:48.674 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.671+0000 7fe5d4967640 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2026-02-13T07:03:48.674 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.671+0000 7fe5d4967640 -1 AuthRegistry(0x7fe5cc0530e0) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx 2026-02-13T07:03:48.675 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.671+0000 7fe5d4967640 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2026-02-13T07:03:48.675 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.671+0000 7fe5d4967640 -1 AuthRegistry(0x7fe5d4965fe0) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx 2026-02-13T07:03:48.676 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.673+0000 7fe5d4967640 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2026-02-13T07:03:48.676 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.673+0000 7fe5d4967640 -1 AuthRegistry(0x7fe5cc04fec0) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx 2026-02-13T07:03:48.676 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.673+0000 7fe5d4967640 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2026-02-13T07:03:48.676 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.673+0000 7fe5d4967640 -1 AuthRegistry(0x7fe5cc10b8b0) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx 2026-02-13T07:03:48.676 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.673+0000 7fe5d4967640 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2026-02-13T07:03:48.676 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:48.673+0000 7fe5d4967640 -1 AuthRegistry(0x7fe5cc04fec0) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx 2026-02-13T07:03:48.725 INFO:tasks.workunit.client.0.trial040.stdout: cluster: 2026-02-13T07:03:48.725 INFO:tasks.workunit.client.0.trial040.stdout: id: 1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: health: HEALTH_OK 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: services: 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: mon: 1 daemons, quorum a (age 7s) [leader: a] 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: mgr: no daemons active 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: osd: 3 osds: 3 up (since 1.99484s), 3 in (since 3s) 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: data: 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: pools: 0 pools, 0 pgs 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: objects: 0 objects, 0 B 2026-02-13T07:03:48.726 INFO:tasks.workunit.client.0.trial040.stdout: usage: 0 B used, 0 B / 0 B avail 2026-02-13T07:03:48.727 INFO:tasks.workunit.client.0.trial040.stdout: pgs: 2026-02-13T07:03:48.727 INFO:tasks.workunit.client.0.trial040.stdout: 2026-02-13T07:03:48.731 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:507: test_run_mon: kill_daemons td/ceph-helpers 2026-02-13T07:03:48.732 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:48.732 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:48.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:48.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:48.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:54.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:54.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:509: test_run_mon: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:03:54.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:03:54.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:03:54.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:03:54.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:03:54.045 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:03:54.046 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:03:54.061 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:54.059+0000 7fb0b6626d80 -1 'td/ceph-helpers/a' already exists and is not empty: monitor may already exist 2026-02-13T07:03:54.063 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:03:54.063 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:54.063 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:54.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:54.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:54.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:03:54.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:03:54.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:03:54.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:54.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:54.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:03:54.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:03:54.095 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:54.095 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:54.095 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:54.095 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:54.095 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.095 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.095 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:54.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:54.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:03:54.138 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:03:54.138 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:54.138 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:54.138 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:03:54.139 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:03:54.139 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:54.139 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:54.140 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:54.140 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:54.140 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.140 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.140 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:54.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:54.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:03:54.185 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:511: test_run_mon: get_asok_path mon.a 2026-02-13T07:03:54.185 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:54.185 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:54.185 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:54.185 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.185 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.185 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:54.185 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:511: test_run_mon: CEPH_ARGS= 2026-02-13T07:03:54.185 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:511: test_run_mon: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get osd_pool_default_size 2026-02-13T07:03:54.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:511: test_run_mon: local 'size={"osd_pool_default_size":"1"}' 2026-02-13T07:03:54.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:512: test_run_mon: test '{"osd_pool_default_size":"1"}' = '{"osd_pool_default_size":"1"}' 2026-02-13T07:03:54.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:513: test_run_mon: kill_daemons td/ceph-helpers 2026-02-13T07:03:54.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:54.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:54.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:54.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:54.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:54.336 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:54.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:515: test_run_mon: CEPH_ARGS='--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd_pool_default_size=2' 2026-02-13T07:03:54.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:515: test_run_mon: run_mon td/ceph-helpers a 2026-02-13T07:03:54.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:03:54.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:03:54.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:03:54.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:03:54.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:03:54.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T07:03:54.352 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:54.350+0000 7f32e081cd80 -1 'td/ceph-helpers/a' already exists and is not empty: monitor may already exist 2026-02-13T07:03:54.355 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:03:54.355 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:54.355 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:54.355 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:54.355 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.355 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.355 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:54.356 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T07:03:54.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:03:54.384 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:03:54.384 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:54.384 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:54.385 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:03:54.385 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:03:54.386 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:54.386 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:54.386 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:54.386 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:54.386 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.386 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.386 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:54.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:54.387 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:03:54.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:03:54.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:54.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:54.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:03:54.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:03:54.431 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:54.432 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:54.432 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:54.432 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:54.432 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.432 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.432 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:54.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:54.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:03:54.476 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:518: test_run_mon: get_asok_path mon.a 2026-02-13T07:03:54.476 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:54.476 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:54.476 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:54.476 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.476 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.476 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:54.477 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:518: test_run_mon: CEPH_ARGS= 2026-02-13T07:03:54.477 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:518: test_run_mon: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get osd_pool_default_size 2026-02-13T07:03:54.520 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:518: test_run_mon: local 'size={"osd_pool_default_size":"2"}' 2026-02-13T07:03:54.521 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:519: test_run_mon: test '{"osd_pool_default_size":"2"}' = '{"osd_pool_default_size":"2"}' 2026-02-13T07:03:54.521 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:520: test_run_mon: kill_daemons td/ceph-helpers 2026-02-13T07:03:54.521 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:54.522 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:54.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:54.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:54.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:54.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:54.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:522: test_run_mon: teardown td/ceph-helpers 2026-02-13T07:03:54.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:03:54.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:03:54.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:03:54.627 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:54.627 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:54.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:54.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:54.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:54.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:54.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:03:54.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:03:54.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:03:54.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:03:54.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:03:54.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:03:54.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:54.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:03:54.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:03:54.637 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:54.637 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:03:54.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:03:54.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:03:54.644 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:03:54.644 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.644 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:03:54.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:03:54.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:03:54.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:03:54.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_run_osd td/ceph-helpers 2026-02-13T07:03:54.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:736: test_run_osd: local dir=td/ceph-helpers 2026-02-13T07:03:54.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:738: test_run_osd: setup td/ceph-helpers 2026-02-13T07:03:54.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:03:54.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:03:54.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:03:54.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:03:54.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:03:54.647 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:03:54.647 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:03:54.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:03:54.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:03:54.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:03:54.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:03:54.648 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:03:54.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:03:54.648 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:03:54.649 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:03:54.649 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:03:54.649 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:03:54.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:54.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:03:54.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:03:54.651 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:03:54.651 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:03:54.652 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:03:54.652 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:03:54.653 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:03:54.653 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.653 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.654 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:03:54.654 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:03:54.654 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:03:54.654 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:03:54.655 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:03:54.656 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.656 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:03:54.657 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:03:54.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:03:54.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:03:54.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:03:54.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:740: test_run_osd: run_mon td/ceph-helpers a 2026-02-13T07:03:54.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:03:54.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:03:54.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:03:54.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:03:54.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:03:54.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T07:03:54.678 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:03:54.678 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:54.678 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:54.678 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:54.678 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.678 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.678 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:54.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T07:03:54.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:03:54.705 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:03:54.705 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:54.705 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:54.705 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:03:54.705 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:03:54.707 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:54.707 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:54.707 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:54.707 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:54.707 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.707 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.707 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:54.707 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:54.707 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:03:54.753 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:03:54.753 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:03:54.754 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:03:54.755 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:03:54.797 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:741: test_run_osd: run_mgr td/ceph-helpers x 2026-02-13T07:03:54.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T07:03:54.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:03:54.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:03:54.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:03:54.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T07:03:54.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:03:54.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:03:54.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:54.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:54.899 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:54.899 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.899 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.899 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:54.900 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:03:54.900 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:03:54.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:743: test_run_osd: run_osd td/ceph-helpers 0 2026-02-13T07:03:54.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:03:54.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:03:54.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:03:54.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:03:54.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:54.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:03:54.919 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:03:54.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=afef7a98-062f-4f15-9764-dd07bbbf81d9 2026-02-13T07:03:54.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 afef7a98-062f-4f15-9764-dd07bbbf81d9' 2026-02-13T07:03:54.920 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 afef7a98-062f-4f15-9764-dd07bbbf81d9 2026-02-13T07:03:54.920 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:03:54.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDazI5pBzWHNxAA1bItsuLCDIiT+2oY5nLx+A== 2026-02-13T07:03:54.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDazI5pBzWHNxAA1bItsuLCDIiT+2oY5nLx+A=="}' 2026-02-13T07:03:54.933 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new afef7a98-062f-4f15-9764-dd07bbbf81d9 -i td/ceph-helpers/0/new.json 2026-02-13T07:03:55.043 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:03:55.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T07:03:55.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDazI5pBzWHNxAA1bItsuLCDIiT+2oY5nLx+A== --osd-uuid afef7a98-062f-4f15-9764-dd07bbbf81d9 2026-02-13T07:03:55.069 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:55.066+0000 7f5e22b1a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:55.070 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:55.068+0000 7f5e22b1a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:55.071 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:55.069+0000 7f5e22b1a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:55.071 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:55.069+0000 7f5e22b1a900 -1 bdev(0x562901d02c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:03:55.071 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:55.069+0000 7f5e22b1a900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T07:03:55.508 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T07:03:55.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:03:55.509 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:03:55.510 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:03:55.510 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:03:55.616 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:03:55.617 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:03:55.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:03:55.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:03:55.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:03:55.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:03:55.633 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:55.631+0000 7f2e1de0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:55.634 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:55.632+0000 7f2e1de0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:55.635 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:55.633+0000 7f2e1de0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:55.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:03:55.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:03:55.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:03:55.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:03:55.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:03:55.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:55.717 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:03:55.718 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:03:55.718 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:55.718 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:03:55.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:03:55.861 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:55.859+0000 7f2e1de0f900 -1 Falling back to public interface 2026-02-13T07:03:56.173 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:56.171+0000 7f2e1de0f900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:03:56.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:03:56.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:56.819 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:03:56.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:03:56.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:56.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:03:56.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:03:57.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:03:57.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:57.920 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:03:57.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-13T07:03:57.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:57.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:03:58.148 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1896765211,v1:127.0.0.1:6801/1896765211] [v2:127.0.0.1:6802/1896765211,v1:127.0.0.1:6803/1896765211] exists,up afef7a98-062f-4f15-9764-dd07bbbf81d9 2026-02-13T07:03:58.149 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:03:58.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:03:58.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:03:58.150 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:745: test_run_osd: get_asok_path osd.0 2026-02-13T07:03:58.150 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T07:03:58.150 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T07:03:58.150 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:03:58.150 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:58.150 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:58.150 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.0.asok 2026-02-13T07:03:58.151 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:745: test_run_osd: CEPH_ARGS= 2026-02-13T07:03:58.151 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:745: test_run_osd: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T07:03:58.198 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:745: test_run_osd: local 'backfills={"osd_max_backfills":"1"}' 2026-02-13T07:03:58.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:746: test_run_osd: echo '{"osd_max_backfills":"1"}' 2026-02-13T07:03:58.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:746: test_run_osd: grep --quiet osd_max_backfills 2026-02-13T07:03:58.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:748: test_run_osd: run_osd td/ceph-helpers 1 --osd-max-backfills 20 2026-02-13T07:03:58.203 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T07:03:58.204 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:03:58.205 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-max-backfills 20' 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:03:58.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6aa431a5-1826-4149-8131-19371563cb6e 2026-02-13T07:03:58.207 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 6aa431a5-1826-4149-8131-19371563cb6e 2026-02-13T07:03:58.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6aa431a5-1826-4149-8131-19371563cb6e' 2026-02-13T07:03:58.207 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:03:58.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDezI5pXdSoDBAA68msl/3zcwRnbjXDG1KzdQ== 2026-02-13T07:03:58.215 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDezI5pXdSoDBAA68msl/3zcwRnbjXDG1KzdQ=="}' 2026-02-13T07:03:58.215 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6aa431a5-1826-4149-8131-19371563cb6e -i td/ceph-helpers/1/new.json 2026-02-13T07:03:58.442 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:03:58.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T07:03:58.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 --mkfs --key AQDezI5pXdSoDBAA68msl/3zcwRnbjXDG1KzdQ== --osd-uuid 6aa431a5-1826-4149-8131-19371563cb6e 2026-02-13T07:03:58.469 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:58.467+0000 7f3811173900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:58.470 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:58.468+0000 7f3811173900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:58.471 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:58.469+0000 7f3811173900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:58.472 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:58.470+0000 7f3811173900 -1 bdev(0x55a322b6ac00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:03:58.472 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:58.470+0000 7f3811173900 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T07:03:58.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T07:03:58.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:03:58.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:03:58.793 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:03:58.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:03:59.104 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:03:59.105 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:03:59.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-13T07:03:59.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:03:59.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:03:59.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:03:59.122 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:59.120+0000 7f6089940900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:59.123 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:59.121+0000 7f6089940900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:59.124 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:59.122+0000 7f6089940900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:03:59.257 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:59.255+0000 7f6089940900 -1 Falling back to public interface 2026-02-13T07:03:59.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:03:59.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:03:59.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:03:59.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:03:59.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:03:59.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:03:59.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:03:59.331 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:03:59.331 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:03:59.331 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:03:59.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:03:59.571 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:03:59.569+0000 7f6089940900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:04:00.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:00.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:00.560 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:00.561 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:00.561 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:00.561 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:00.791 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2523211459,v1:127.0.0.1:6811/2523211459] [v2:127.0.0.1:6812/2523211459,v1:127.0.0.1:6813/2523211459] exists,up 6aa431a5-1826-4149-8131-19371563cb6e 2026-02-13T07:04:00.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:00.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:00.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:00.793 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:750: test_run_osd: get_asok_path osd.1 2026-02-13T07:04:00.793 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T07:04:00.793 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T07:04:00.793 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:00.793 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:00.793 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:00.793 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.1.asok 2026-02-13T07:04:00.794 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:750: test_run_osd: CEPH_ARGS= 2026-02-13T07:04:00.794 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:750: test_run_osd: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.1.asok config get osd_max_backfills 2026-02-13T07:04:00.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:750: test_run_osd: local 'backfills={"osd_max_backfills":"20"}' 2026-02-13T07:04:00.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:751: test_run_osd: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-13T07:04:00.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:753: test_run_osd: CEPH_ARGS='--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-max-backfills 30' 2026-02-13T07:04:00.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:753: test_run_osd: run_osd td/ceph-helpers 2 2026-02-13T07:04:00.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:00.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:00.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T07:04:00.843 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/2 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-max-backfills 30' 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/2' 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/2/journal' 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:00.844 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:00.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:00.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:00.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:00.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:00.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:00.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/2 2026-02-13T07:04:00.846 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:00.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3f472539-4ed5-4742-89ce-480c2f6d9d8a 2026-02-13T07:04:00.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 3f472539-4ed5-4742-89ce-480c2f6d9d8a' 2026-02-13T07:04:00.846 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 3f472539-4ed5-4742-89ce-480c2f6d9d8a 2026-02-13T07:04:00.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:00.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgzI5pFwDcMhAAWVP59ROSOkC6LKHpt005Ng== 2026-02-13T07:04:00.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgzI5pFwDcMhAAWVP59ROSOkC6LKHpt005Ng=="}' 2026-02-13T07:04:00.856 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3f472539-4ed5-4742-89ce-480c2f6d9d8a -i td/ceph-helpers/2/new.json 2026-02-13T07:04:01.086 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:04:01.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/2/new.json 2026-02-13T07:04:01.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-max-backfills 30 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/2 --osd-journal=td/ceph-helpers/2/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgzI5pFwDcMhAAWVP59ROSOkC6LKHpt005Ng== --osd-uuid 3f472539-4ed5-4742-89ce-480c2f6d9d8a 2026-02-13T07:04:01.114 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:01.112+0000 7f910c00d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:01.115 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:01.113+0000 7f910c00d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:01.116 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:01.114+0000 7f910c00d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:01.117 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:01.114+0000 7f910c00d900 -1 bdev(0x5593fcb12c00 td/ceph-helpers/2/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:01.117 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:01.115+0000 7f910c00d900 -1 bluestore(td/ceph-helpers/2) _read_fsid unparsable uuid 2026-02-13T07:04:01.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/2/keyring 2026-02-13T07:04:01.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:01.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T07:04:01.432 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T07:04:01.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:01.748 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:04:01.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T07:04:01.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-max-backfills 30 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/2 --osd-journal=td/ceph-helpers/2/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:01.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:01.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:01.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:01.766 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:01.764+0000 7f0d8221b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:01.767 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:01.765+0000 7f0d8221b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:01.768 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:01.766+0000 7f0d8221b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:01.905 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:01.903+0000 7f0d8221b900 -1 Falling back to public interface 2026-02-13T07:04:01.983 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T07:04:01.983 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:01.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:04:01.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:01.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:01.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:01.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:01.984 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:01.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:01.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:04:02.204 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:02.201+0000 7f0d8221b900 -1 osd.2 0 log_to_monitors true 2026-02-13T07:04:02.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:03.217 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:03.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:03.218 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:03.218 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:03.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:03.219 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:04:03.444 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3370449732,v1:127.0.0.1:6819/3370449732] [v2:127.0.0.1:6820/3370449732,v1:127.0.0.1:6821/3370449732] exists,up 3f472539-4ed5-4742-89ce-480c2f6d9d8a 2026-02-13T07:04:03.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:03.445 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:03.445 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:03.446 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:755: test_run_osd: get_asok_path osd.2 2026-02-13T07:04:03.446 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-13T07:04:03.446 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-13T07:04:03.446 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:03.446 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.446 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.446 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-osd.2.asok 2026-02-13T07:04:03.447 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:755: test_run_osd: CEPH_ARGS= 2026-02-13T07:04:03.447 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:755: test_run_osd: ceph --format=json daemon /tmp/ceph-asok.63910/ceph-osd.2.asok config get osd_max_backfills 2026-02-13T07:04:03.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:755: test_run_osd: local 'backfills={"osd_max_backfills":"30"}' 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:756: test_run_osd: test '{"osd_max_backfills":"30"}' = '{"osd_max_backfills":"30"}' 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:758: test_run_osd: teardown td/ceph-helpers 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:03.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:03.602 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:03.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:03.603 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:03.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:03.605 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:03.605 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:03.606 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:03.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:03.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:03.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:03.607 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:03.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:03.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:03.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:03.614 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:03.615 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.615 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:03.616 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_set_config td/ceph-helpers 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: test_set_config: local dir=td/ceph-helpers 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1165: test_set_config: setup td/ceph-helpers 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:03.617 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:03.618 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:03.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:03.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:03.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:03.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:03.618 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:03.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:03.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:03.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:03.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:03.620 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:03.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:03.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:03.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:03.621 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:03.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:03.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:03.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:03.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:03.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:03.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:03.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:03.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:04:03.626 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:04:03.627 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.627 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:04:03.627 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:04:03.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:04:03.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:04:03.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:04:03.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1166: test_set_config: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:04:03.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:04:03.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:04:03.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:04:03.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:04:03.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:04:03.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:04:03.649 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:04:03.650 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:03.650 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:03.650 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:03.650 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.650 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.650 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:03.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:04:03.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:04:03.679 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:04:03.679 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:03.679 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:03.679 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:04:03.679 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:04:03.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:03.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:03.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:03.681 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:03.681 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.681 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.681 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:03.681 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:03.681 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:04:03.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:04:03.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:03.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:03.726 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:04:03.727 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:04:03.727 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:03.727 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:03.728 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:03.728 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:03.728 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.728 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.728 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:03.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:03.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:04:03.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1167: test_set_config: get_config mon a ms_crc_header 2026-02-13T07:04:03.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:03.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:03.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=ms_crc_header 2026-02-13T07:04:03.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .ms_crc_header 2026-02-13T07:04:03.775 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:03.775 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:03.776 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:03.776 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:03.776 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.776 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.776 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:03.776 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:03.776 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get ms_crc_header 2026-02-13T07:04:03.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1167: test_set_config: test true = true 2026-02-13T07:04:03.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: test_set_config: set_config mon a ms_crc_header false 2026-02-13T07:04:03.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=mon 2026-02-13T07:04:03.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=a 2026-02-13T07:04:03.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=ms_crc_header 2026-02-13T07:04:03.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=false 2026-02-13T07:04:03.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T07:04:03.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path mon.a 2026-02-13T07:04:03.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:03.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:03.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:03.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:03.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config set ms_crc_header false 2026-02-13T07:04:03.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T07:04:03.866 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1169: test_set_config: get_config mon a ms_crc_header 2026-02-13T07:04:03.866 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:03.866 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:03.866 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=ms_crc_header 2026-02-13T07:04:03.866 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .ms_crc_header 2026-02-13T07:04:03.867 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:03.867 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:03.868 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:03.868 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:03.868 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.868 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.868 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:03.868 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:03.868 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get ms_crc_header 2026-02-13T07:04:03.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1169: test_set_config: test false = false 2026-02-13T07:04:03.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1170: test_set_config: set_config mon a ms_crc_header true 2026-02-13T07:04:03.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=mon 2026-02-13T07:04:03.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=a 2026-02-13T07:04:03.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=ms_crc_header 2026-02-13T07:04:03.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T07:04:03.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T07:04:03.913 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path mon.a 2026-02-13T07:04:03.913 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:03.913 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:03.913 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:03.913 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.913 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.913 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:03.913 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config set ms_crc_header true 2026-02-13T07:04:03.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T07:04:03.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1171: test_set_config: get_config mon a ms_crc_header 2026-02-13T07:04:03.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:03.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:03.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=ms_crc_header 2026-02-13T07:04:03.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .ms_crc_header 2026-02-13T07:04:03.959 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:03.959 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:03.959 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:03.959 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:03.959 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:03.959 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:03.959 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:03.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:03.960 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get ms_crc_header 2026-02-13T07:04:04.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1171: test_set_config: test true = true 2026-02-13T07:04:04.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1172: test_set_config: teardown td/ceph-helpers 2026-02-13T07:04:04.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:04.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:04.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:04.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:04.002 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:04.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:04.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:04.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:04.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:04.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:04.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:04.109 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:04.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:04.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:04.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:04.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:04.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:04.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:04.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:04.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:04.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:04.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:04.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:04.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:04.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:04.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:04.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_setup td/ceph-helpers 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: test_setup: local dir=td/ceph-helpers 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: test_setup: setup td/ceph-helpers 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:04.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:04.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:04.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:04.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:04.119 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:04.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:04.120 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:04.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:04.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:04.121 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:04.121 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.122 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:04.122 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:04.122 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:04.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:04.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:04.125 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:04.125 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:04.125 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:04.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:04.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:04.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:04.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:04:04.127 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:04:04.127 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:04.127 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:04.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:04:04.128 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:04:04.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:137: test_setup: test -d td/ceph-helpers 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: test_setup: setup td/ceph-helpers 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:04.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:04.131 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:04.131 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:04.131 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:04.131 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:04.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:04.132 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:04.133 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:04.133 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:04.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:04.133 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:04.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:04.134 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.135 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:04.135 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:04.136 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:04.137 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:04.137 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:04.137 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:04.137 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:04.138 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:04.138 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:04.138 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:04:04.139 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:04:04.139 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:04.139 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:04.139 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:04:04.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:04:04.141 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: test_setup: test -d td/ceph-helpers 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:140: test_setup: teardown td/ceph-helpers 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:04.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:04.143 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:04.143 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:04.143 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:04.144 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:04.145 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:04.145 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:04.145 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:04.146 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.146 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:04.146 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:04.146 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.147 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:04.148 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:04.148 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:04.149 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:04.149 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:04.149 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:04.149 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:04.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:04.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:04.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:04:04.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_teardown td/ceph-helpers 2026-02-13T07:04:04.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:211: test_teardown: local dir=td/ceph-helpers 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:212: test_teardown: setup td/ceph-helpers 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:04.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:04.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:04.153 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:04.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:04.154 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:04.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:04.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:04.155 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:04.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:04.156 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:04.156 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:04.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:04.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:04.159 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:04.159 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:04.159 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:04.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:04.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:04.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:04.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:04:04.161 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:04:04.161 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:04.161 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:04.161 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:04:04.162 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:04:04.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:213: test_teardown: teardown td/ceph-helpers 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:04.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:04.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:04.165 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:04.166 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:04.166 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:04.167 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:04.167 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:04.167 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:04.168 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.168 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:04.168 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:04.169 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:04.169 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:04.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:04.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:04.172 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:04.172 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:04.172 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:04.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:04.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:04.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:214: test_teardown: test -d td/ceph-helpers 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_wait_background td/ceph-helpers 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: test_wait_background: local pids= 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2238: test_wait_background: run_in_background pids bash -c 'sleep 1; exit 1' 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 129225"' 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 129225' 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: test_wait_background: run_in_background pids bash -c 'sleep 2; exit 0' 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-13T07:04:04.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 129227"' 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 129227' 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2240: test_wait_background: wait_background pids 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 129225 129227' 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: bash -c 'sleep 1; exit 1' 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 129225 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/129228: /' 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: bash -c 'sleep 2; exit 0' 2026-02-13T07:04:04.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/129230: /' 2026-02-13T07:04:05.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 1 2026-02-13T07:04:05.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2225: wait_background: return_code=1 2026-02-13T07:04:05.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-13T07:04:05.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 129227 2026-02-13T07:04:06.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-13T07:04:06.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-13T07:04:06.178 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-13T07:04:06.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 1 2026-02-13T07:04:06.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2241: test_wait_background: '[' 1 -ne 1 ']' 2026-02-13T07:04:06.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2243: test_wait_background: run_in_background pids bash -c 'sleep 1; exit 0' 2026-02-13T07:04:06.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-13T07:04:06.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 129233"' 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 129233' 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2244: test_wait_background: run_in_background pids bash -c 'sleep 2; exit 0' 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 129234"' 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 129234' 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2245: test_wait_background: wait_background pids 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 129233 129234' 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 129233 2026-02-13T07:04:06.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: bash -c 'sleep 1; exit 0' 2026-02-13T07:04:06.180 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/129236: /' 2026-02-13T07:04:06.180 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: bash -c 'sleep 2; exit 0' 2026-02-13T07:04:06.180 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/129238: /' 2026-02-13T07:04:07.180 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-13T07:04:07.180 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-13T07:04:07.180 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 129234 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2246: test_wait_background: '[' 0 -ne 0 ']' 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2248: test_wait_background: '[' '!' -z '' ']' 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_wait_for_clean td/ceph-helpers 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1682: test_wait_for_clean: local dir=td/ceph-helpers 2026-02-13T07:04:08.182 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: test_wait_for_clean: setup td/ceph-helpers 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:08.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:08.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:08.184 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:08.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:08.185 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:08.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:08.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:08.187 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:08.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:08.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:08.188 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:08.188 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:08.188 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:08.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:08.189 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:08.190 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:08.191 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:08.191 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:08.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:08.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:08.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:08.192 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:04:08.193 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:04:08.193 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:08.193 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:08.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: test_wait_for_clean: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:04:08.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:04:08.196 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-13T07:04:08.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:04:08.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:08.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:08.215 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:08.215 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:08.215 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:08.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:08.216 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-13T07:04:08.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:04:08.243 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:04:08.243 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:08.243 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:08.243 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:04:08.243 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:04:08.244 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:08.244 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:08.244 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:08.244 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:08.245 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:08.245 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:08.245 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:08.245 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:08.245 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:04:08.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:04:08.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:08.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:08.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:04:08.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:04:08.290 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:08.290 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:08.291 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:08.291 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:08.291 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:08.291 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:08.291 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:08.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:08.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:04:08.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1686: test_wait_for_clean: run_osd td/ceph-helpers 0 2026-02-13T07:04:08.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:08.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:08.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:08.341 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:08.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f98ed8c8-16dc-46af-a4e4-868ae51762dd 2026-02-13T07:04:08.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f98ed8c8-16dc-46af-a4e4-868ae51762dd' 2026-02-13T07:04:08.344 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 f98ed8c8-16dc-46af-a4e4-868ae51762dd 2026-02-13T07:04:08.344 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:08.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDozI5p5PbdFBAAPVaOKztrlhZCVBI9B7vBBw== 2026-02-13T07:04:08.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDozI5p5PbdFBAAPVaOKztrlhZCVBI9B7vBBw=="}' 2026-02-13T07:04:08.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f98ed8c8-16dc-46af-a4e4-868ae51762dd -i td/ceph-helpers/0/new.json 2026-02-13T07:04:08.457 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:08.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T07:04:08.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDozI5p5PbdFBAAPVaOKztrlhZCVBI9B7vBBw== --osd-uuid f98ed8c8-16dc-46af-a4e4-868ae51762dd 2026-02-13T07:04:08.482 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:08.480+0000 7f9435ed5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:08.483 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:08.481+0000 7f9435ed5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:08.484 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:08.482+0000 7f9435ed5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:08.485 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:08.483+0000 7f9435ed5900 -1 bdev(0x5594efdf4c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:08.485 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:08.483+0000 7f9435ed5900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T07:04:08.804 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T07:04:08.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:08.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:04:08.806 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:04:08.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:08.910 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:04:08.910 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:04:08.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:08.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:08.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:08.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:08.927 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:08.925+0000 7fca1800d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:08.929 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:08.927+0000 7fca1800d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:08.930 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:08.928+0000 7fca1800d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:09.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:04:09.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:09.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:04:09.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:09.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:09.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:09.013 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:09.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:09.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:09.013 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:09.101 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:09.099+0000 7fca1800d900 -1 Falling back to public interface 2026-02-13T07:04:09.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:09.360 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:09.358+0000 7fca1800d900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:04:10.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:10.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:10.113 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:10.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:10.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:10.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:10.212 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3754245100,v1:127.0.0.1:6801/3754245100] [v2:127.0.0.1:6802/3754245100,v1:127.0.0.1:6803/3754245100] exists,up f98ed8c8-16dc-46af-a4e4-868ae51762dd 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: test_wait_for_clean: run_mgr td/ceph-helpers x 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T07:04:10.214 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:04:10.313 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:04:10.314 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:10.314 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:10.314 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:10.315 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:10.315 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:10.315 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:10.315 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:04:10.315 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:04:10.333 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1688: test_wait_for_clean: create_rbd_pool 2026-02-13T07:04:10.333 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T07:04:10.430 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T07:04:10.437 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T07:04:10.437 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T07:04:10.611 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T07:04:10.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T07:04:11.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T07:04:11.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1689: test_wait_for_clean: WAIT_FOR_CLEAN_TIMEOUT=1 2026-02-13T07:04:11.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1689: test_wait_for_clean: wait_for_clean 2026-02-13T07:04:11.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:04:11.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:04:11.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:04:11.898 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 1 .1 2026-02-13T07:04:11.899 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:04:11.899 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:04:11.899 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:04:11.899 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:04:11.899 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:04:11.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.3') 2026-02-13T07:04:11.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:04:11.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:04:11.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:04:11.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:04:11.923 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:04:12.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T07:04:12.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:04:12.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:04:12.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:04:12.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=0 2026-02-13T07:04:12.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 0 2026-02-13T07:04:12.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-0' 2026-02-13T07:04:12.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:04:12.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-0 2026-02-13T07:04:12.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:04:12.096 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:04:12.096 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-0 2026-02-13T07:04:12.096 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:04:12.097 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=0 2026-02-13T07:04:12.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 0' 2026-02-13T07:04:12.098 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 0 2026-02-13T07:04:12.098 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:04:12.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 0 -lt 0 2026-02-13T07:04:12.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:04:12.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:12.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:12.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 0 == 0 2026-02-13T07:04:12.303 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1655: wait_for_clean: sleep 1 2026-02-13T07:04:13.305 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:04:13.305 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:13.306 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:13.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T07:04:13.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:13.618 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:13.618 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:13.618 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:13.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:13.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:13.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:13.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:13.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:13.840 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:13.840 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:14.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:14.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-13T07:04:14.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-13T07:04:14.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-13T07:04:14.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T07:04:14.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-13T07:04:14.253 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T07:04:14.254 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:14.254 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:14.254 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:14.254 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:14.254 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:14.254 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:14.254 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:14.472 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:14.473 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:14.473 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:14.473 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:14.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:14.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-13T07:04:14.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-13T07:04:14.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T07:04:14.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T07:04:14.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T07:04:14.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T07:04:14.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T07:04:14.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T07:04:15.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T07:04:15.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T07:04:15.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 4 )) 2026-02-13T07:04:15.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T07:04:15.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-13T07:04:15.306 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T07:04:15.306 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:15.307 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:15.307 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:15.307 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:15.308 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:15.308 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:15.308 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:15.529 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:15.530 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:15.530 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:15.530 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:15.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:15.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-13T07:04:15.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-13T07:04:15.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T07:04:15.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T07:04:15.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T07:04:15.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T07:04:15.849 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T07:04:15.849 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T07:04:16.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T07:04:16.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T07:04:16.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 4 )) 2026-02-13T07:04:16.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T07:04:16.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-13T07:04:16.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T07:04:16.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:16.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:16.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:16.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:16.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:16.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:16.559 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:16.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:16.785 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:16.785 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:16.785 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:17.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:17.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-13T07:04:17.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-13T07:04:17.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T07:04:17.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T07:04:17.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T07:04:17.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T07:04:17.099 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T07:04:17.099 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T07:04:17.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T07:04:17.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T07:04:17.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 4 )) 2026-02-13T07:04:17.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T07:04:17.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.3 2026-02-13T07:04:17.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T07:04:17.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:17.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:17.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:17.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:17.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:17.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:17.712 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:17.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:17.932 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:17.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:17.933 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:18.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:18.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-13T07:04:18.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-13T07:04:18.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T07:04:18.244 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T07:04:18.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T07:04:18.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T07:04:18.245 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T07:04:18.245 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T07:04:18.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T07:04:18.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T07:04:18.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 4 )) 2026-02-13T07:04:18.561 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-13T07:04:18.867 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T07:04:18.867 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_fingerprint": "0dfe2d81-68cf-42ef-9efd-36bb2a16e13c", 2026-02-13T07:04:18.867 INFO:tasks.workunit.client.0.trial040.stdout: "version": "20.2.0-636-g3b40640f", 2026-02-13T07:04:18.867 INFO:tasks.workunit.client.0.trial040.stdout: "commit": "3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb", 2026-02-13T07:04:18.867 INFO:tasks.workunit.client.0.trial040.stdout: "timestamp": "2026-02-13T07:04:18.864971+0000", 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "tag": "", 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "health": { 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "status": "HEALTH_WARN", 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "checks": { 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "PG_DEGRADED": { 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "message": "Degraded data redundancy: 2/4 objects degraded (50.000%), 2 pgs degraded", 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "count": 2 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [ 2026-02-13T07:04:18.868 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0]" 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0]" 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: "TOO_FEW_OSDS": { 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:04:18.869 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: "message": "OSD count 1 < osd_pool_default_size 3", 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: "count": 2 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [], 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: "mutes": [] 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: "monmap_first_committed": 1, 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: "monmap_last_committed": 1, 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: "monmap": { 2026-02-13T07:04:18.870 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 1, 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "fsid": "1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2", 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "modified": "2026-02-13T07:04:08.206933Z", 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "created": "2026-02-13T07:04:08.206933Z", 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "min_mon_release": 20, 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "min_mon_release_name": "tentacle", 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "election_strategy": 1, 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "disallowed_leaders": "", 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode": false, 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "tiebreaker_mon": "", 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "removed_ranks": "", 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "features": { 2026-02-13T07:04:18.871 INFO:tasks.workunit.client.0.trial040.stdout: "persistent": [ 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "kraken", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "luminous", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "mimic", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap-prune", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "nautilus", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "octopus", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "pacific", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "elector-pinging", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "quincy", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "reef", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "squid", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "tentacle", 2026-02-13T07:04:18.872 INFO:tasks.workunit.client.0.trial040.stdout: "nvmeof_beacon_diff" 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: "optional": [] 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: "mons": [ 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: "rank": 0, 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: "name": "a", 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: "public_addrs": { 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:18.873 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:7109", 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 0 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:7109/0", 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: "public_addr": "127.0.0.1:7109/0", 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: "priority": 0, 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 0, 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: "crush_location": "{}" 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.874 INFO:tasks.workunit.client.0.trial040.stdout: "quorum": [ 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: 0 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap": { 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 11, 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "fsid": "1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2", 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "created": "2026-02-13T07:04:08.242278+0000", 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "modified": "2026-02-13T07:04:11.890452+0000", 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "last_up_change": "2026-02-13T07:04:09.960916+0000", 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "last_in_change": "2026-02-13T07:04:08.432697+0000", 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "flags_num": 5799936, 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "flags_set": [ 2026-02-13T07:04:18.875 INFO:tasks.workunit.client.0.trial040.stdout: "pglog_hardlimit", 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "purged_snapdirs", 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "recovery_deletes", 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "sortbitwise" 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "crush_version": 3, 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "full_ratio": 0.99000000953674316, 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_snapshot": "", 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "pool_max": 1, 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "max_osd": 1, 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "require_min_compat_client": "luminous", 2026-02-13T07:04:18.876 INFO:tasks.workunit.client.0.trial040.stdout: "min_compat_client": "jewel", 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "require_osd_release": "tentacle", 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "allow_crimson": false, 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "pools": [ 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "pool_name": "rbd", 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "create_time": "2026-02-13T07:04:10.518623+0000", 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "flags": 8193, 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "type": 1, 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "size": 2, 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "min_size": 1, 2026-02-13T07:04:18.877 INFO:tasks.workunit.client.0.trial040.stdout: "crush_rule": 0, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_count": 0, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_target": 0, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_barrier": 0, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "is_stretch_pool": false, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "object_hash": 2, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "pg_autoscale_mode": "off", 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "pg_num": 4, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "pg_placement_num": 4, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "pg_placement_num_target": 4, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "pg_num_target": 4, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "pg_num_pending": 4, 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "last_pg_merge_meta": { 2026-02-13T07:04:18.878 INFO:tasks.workunit.client.0.trial040.stdout: "source_pgid": "0.0", 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "ready_epoch": 0, 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "last_epoch_started": 0, 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "last_epoch_clean": 0, 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "source_version": "0'0", 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "target_version": "0'0" 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "last_change": "11", 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "last_force_op_resend": "0", 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "last_force_op_resend_preluminous": "0", 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "auid": 0, 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "snap_mode": "selfmanaged", 2026-02-13T07:04:18.879 INFO:tasks.workunit.client.0.trial040.stdout: "snap_seq": 2, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "snap_epoch": 11, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "pool_snaps": [], 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "removed_snaps": "[]", 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "quota_max_bytes": 0, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "quota_max_objects": 0, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "tiers": [], 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "tier_of": -1, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "read_tier": -1, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "write_tier": -1, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "cache_mode": "none", 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "target_max_bytes": 0, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "target_max_objects": 0, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-13T07:04:18.880 INFO:tasks.workunit.client.0.trial040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "cache_min_flush_age": 0, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "cache_min_evict_age": 0, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "erasure_code_profile": "", 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_params": { 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "type": "none" 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_period": 0, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_count": 0, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "use_gmt_hitset": true, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "min_read_recency_for_promote": 0, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "min_write_recency_for_promote": 0, 2026-02-13T07:04:18.881 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_grade_decay_rate": 0, 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_search_last_n": 0, 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "grade_table": [], 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "stripe_width": 0, 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "expected_num_objects": 0, 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "fast_read": false, 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "nonprimary_shards": "{}", 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "options": {}, 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "application_metadata": { 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "rbd": {} 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "read_balance": { 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "score_type": "Fair distribution", 2026-02-13T07:04:18.882 INFO:tasks.workunit.client.0.trial040.stdout: "score_acting": 1, 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: "score_stable": 1, 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: "optimal_score": 1, 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: "raw_score_acting": 1, 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: "raw_score_stable": 1, 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: "primary_affinity_weighted": 1, 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: "average_primary_affinity": 1, 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: "average_primary_affinity_weighted": 1 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: "osds": [ 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.883 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "uuid": "f98ed8c8-16dc-46af-a4e4-868ae51762dd", 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "up": 1, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "in": 1, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 1, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "primary_affinity": 1, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "last_clean_begin": 0, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "last_clean_end": 0, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "up_from": 5, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "up_thru": 6, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "down_at": 0, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "lost_at": 0, 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "public_addrs": { 2026-02-13T07:04:18.884 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6800", 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 3754245100 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6801", 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 3754245100 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.885 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_addrs": { 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6802", 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 3754245100 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6803", 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 3754245100 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.886 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_back_addrs": { 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6806", 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 3754245100 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6807", 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 3754245100 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.887 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_front_addrs": { 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6804", 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 3754245100 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6805", 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 3754245100 2026-02-13T07:04:18.888 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: "public_addr": "127.0.0.1:6801/3754245100", 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_addr": "127.0.0.1:6803/3754245100", 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_back_addr": "127.0.0.1:6807/3754245100", 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_front_addr": "127.0.0.1:6805/3754245100", 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: "state": [ 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: "exists", 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: "up" 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.889 INFO:tasks.workunit.client.0.trial040.stdout: "osd_xinfo": [ 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "down_stamp": "0.000000", 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "laggy_probability": 0, 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "laggy_interval": 0, 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "features": 4544132024016699391, 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "old_weight": 0, 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "dead_epoch": 0 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "pg_upmap": [], 2026-02-13T07:04:18.890 INFO:tasks.workunit.client.0.trial040.stdout: "pg_upmap_items": [], 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "pg_upmap_primaries": [], 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "pg_temp": [], 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "primary_temp": [], 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "blocklist": {}, 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "range_blocklist": {}, 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "erasure_code_profiles": { 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "default": { 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "k": "2", 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "m": "2", 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "plugin": "isa", 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: "technique": "reed_sol_van" 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.891 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: "removed_snaps_queue": [ 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: "snaps": [ 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: "begin": 2, 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: "length": 1 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: "new_removed_snaps": [ 2026-02-13T07:04:18.892 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: "snaps": [ 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: "begin": 2, 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: "length": 1 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: "new_purged_snaps": [], 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: "crush_node_flags": {}, 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: "device_class_flags": {}, 2026-02-13T07:04:18.893 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode": { 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode_enabled": false, 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_bucket_count": 0, 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: "degraded_stretch_mode": 0, 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: "recovering_stretch_mode": 0, 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode_bucket": 0 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: "osd_metadata": [ 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: "arch": "x86_64", 2026-02-13T07:04:18.894 INFO:tasks.workunit.client.0.trial040.stdout: "back_addr": "[v2:127.0.0.1:6802/3754245100,v1:127.0.0.1:6803/3754245100]", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "back_iface": "", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs": "1", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs_dedicated_db": "0", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs_dedicated_wal": "0", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs_single_shared_device": "1", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_allocation_from_file": "1", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_allocator": "hybrid", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_access_mode": "file", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_block_size": "4096", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_path": "td/ceph-helpers/0/block", 2026-02-13T07:04:18.895 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_rotational": "0", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_size": "107374182400", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_support_discard": "1", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_type": "ssd", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_min_alloc_size": "4096", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_write_mode": "classic", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_release": "tentacle", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_version": "ceph version 20.2.0-636-g3b40640f (3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb) tentacle (stable - RelWithDebInfo)", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_version_short": "20.2.0-636-g3b40640f", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_version_when_created": "ceph version 20.2.0-636-g3b40640f (3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb) tentacle (stable - RelWithDebInfo)", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "created_at": "2026-02-13T07:04:08.757889Z", 2026-02-13T07:04:18.896 INFO:tasks.workunit.client.0.trial040.stdout: "default_device_class": "ssd", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6793", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "devices": "nvme0n1", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "distro": "centos", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "distro_description": "CentOS Stream 9", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "distro_version": "9", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "front_addr": "[v2:127.0.0.1:6800/3754245100,v1:127.0.0.1:6801/3754245100]", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "front_iface": "", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/3754245100,v1:127.0.0.1:6807/3754245100]", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/3754245100,v1:127.0.0.1:6805/3754245100]", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "hostname": "trial040", 2026-02-13T07:04:18.897 INFO:tasks.workunit.client.0.trial040.stdout: "journal_rotational": "0", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 6 13:57:07 UTC 2026", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "kernel_version": "5.14.0-677.el9.x86_64", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "mem_swap_kb": "8388604", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "mem_total_kb": "130438944", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "objectstore_numa_node": "0", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "objectstore_numa_nodes": "0", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "os": "Linux", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "osd_data": "td/ceph-helpers/0", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "osd_objectstore": "bluestore", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "osdspec_affinity": "", 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: "rotational": "0" 2026-02-13T07:04:18.898 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap_clean_epochs": { 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: "min_last_epoch_clean": 0, 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: "last_epoch_clean": { 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: "per_pool": [] 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: "osd_epochs": [ 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 5 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.899 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap_first_committed": 1, 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap_last_committed": 11, 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: "crushmap": { 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: "devices": [ 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: "name": "osd.0", 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: "class": "ssd" 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: "types": [ 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.900 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 0, 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: "name": "osd" 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 1, 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: "name": "host" 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 2, 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: "name": "chassis" 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 3, 2026-02-13T07:04:18.901 INFO:tasks.workunit.client.0.trial040.stdout: "name": "rack" 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 4, 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: "name": "row" 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 5, 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: "name": "pdu" 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 6, 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: "name": "pod" 2026-02-13T07:04:18.902 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 7, 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: "name": "room" 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 8, 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: "name": "datacenter" 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 9, 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: "name": "zone" 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.903 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 10, 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: "name": "region" 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 11, 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: "name": "root" 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: "buckets": [ 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: "id": -1, 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: "name": "default", 2026-02-13T07:04:18.904 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 11, 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "root", 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: "id": -3, 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.905 INFO:tasks.workunit.client.0.trial040.stdout: "id": -2, 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "name": "default~ssd", 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 11, 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "root", 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "id": -4, 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.906 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "id": -3, 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "name": "trial040", 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 1, 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "host", 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.907 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "id": -4, 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "name": "trial040~ssd", 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 1, 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "host", 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.908 INFO:tasks.workunit.client.0.trial040.stdout: "rules": [ 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "rule_id": 0, 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "rule_name": "replicated_rule", 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "type": 1, 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "steps": [ 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "op": "take", 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "item": -1, 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "item_name": "default" 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "op": "choose_firstn", 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "num": 0, 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: "type": "osd" 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.909 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "op": "emit" 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "tunables": { 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "choose_local_tries": 0, 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "choose_local_fallback_tries": 0, 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "choose_total_tries": 50, 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "chooseleaf_descend_once": 1, 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "chooseleaf_vary_r": 1, 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "chooseleaf_stable": 1, 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "msr_descents": 100, 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "msr_collision_tries": 100, 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "straw_calc_version": 1, 2026-02-13T07:04:18.910 INFO:tasks.workunit.client.0.trial040.stdout: "allowed_bucket_algs": 54, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "profile": "jewel", 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "optimal_tunables": 1, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "legacy_tunables": 0, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "minimum_required_version": "jewel", 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables": 1, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables2": 1, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "has_v2_rules": 0, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables3": 1, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "has_v3_rules": 0, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "has_v4_buckets": 1, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables5": 1, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "has_v5_rules": 0, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "has_msr_rules": 0 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "choose_args": {} 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "fsmap": { 2026-02-13T07:04:18.911 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 1, 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "btime": "2026-02-13T07:04:08:242133+0000", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "default_fscid": -1, 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "compat": { 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "compat": {}, 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "ro_compat": {}, 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "incompat": { 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_1": "base v0.20", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_2": "client writeable ranges", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_3": "default file layouts on dirs", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_4": "dir inode in separate object", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_5": "mds uses versioned encoding", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_8": "no anchor table", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_9": "file layout v2", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_10": "snaprealm v2", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_11": "minor log segments", 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: "feature_12": "quiesce subvolumes" 2026-02-13T07:04:18.912 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "feature_flags": { 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "enable_multiple": true, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "ever_enabled_multiple": true 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "standbys": [], 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "filesystems": [] 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "mdsmap_first_committed": 1, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "mdsmap_last_committed": 1, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "auth": { 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "first_committed": 1, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "last_committed": 3, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "num_secrets": 1 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg": 4, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_active": 4, 2026-02-13T07:04:18.913 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_unknown": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_osd": 1, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "pool_sum": { 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "stat_sum": { 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes": 19, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects": 2, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_clones": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_copies": 4, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_degraded": 2, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_misplaced": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_unfound": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_dirty": 2, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_whiteouts": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_read": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_read_kb": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_write": 2, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_write_kb": 2, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_scrub_errors": 0, 2026-02-13T07:04:18.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_deep_scrub_errors": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_recovered": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_recovered": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_keys_recovered": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_omap": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_kb": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_kb": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_promote": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_high": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_low": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_some": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_full": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_pinned": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_legacy_snapsets": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_large_omap_objects": 0, 2026-02-13T07:04:18.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_manifest": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_bytes": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_keys": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_repaired": 0 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "store_stats": { 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "total": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "available": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 0, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 0 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "log_size": 3, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "ondisk_log_size": 3, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "up": 4, 2026-02-13T07:04:18.916 INFO:tasks.workunit.client.0.trial040.stdout: "acting": 4, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_store_stats": 0 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "osd_sum": { 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "up_from": 0, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "seq": 0, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_pgs": 4, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_osds": 1, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_osds": 1, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_omap_osds": 1, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "kb": 104857600, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used": 27060, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_data": 100, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_omap": 7, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_meta": 26936, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "kb_avail": 104830540, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "statfs": { 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "total": 107374182400, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "available": 107346472960, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 102400, 2026-02-13T07:04:18.917 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 21337, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 8135, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 27582521 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "hb_peers": [], 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "snap_trim_queue_len": 0, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_snap_trimming": 0, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_shards_repaired": 0, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "op_queue_age_hist": { 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "histogram": [], 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "upper_bound": 1 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "perf_stat": { 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ms": 0, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ms": 0, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ns": 0, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ns": 0 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "alerts": [], 2026-02-13T07:04:18.918 INFO:tasks.workunit.client.0.trial040.stdout: "network_ping_times": [] 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "osd_sum_by_class": { 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "ssd": { 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "up_from": 0, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "seq": 0, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "num_pgs": 4, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "num_osds": 1, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_osds": 1, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_omap_osds": 1, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "kb": 104857600, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used": 27060, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_data": 100, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_omap": 7, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_meta": 26936, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "kb_avail": 104830540, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "statfs": { 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "total": 107374182400, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "available": 107346472960, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 102400, 2026-02-13T07:04:18.919 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 21337, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 8135, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 27582521 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "hb_peers": [], 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "snap_trim_queue_len": 0, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "num_snap_trimming": 0, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "num_shards_repaired": 0, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "op_queue_age_hist": { 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "histogram": [], 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "upper_bound": 1 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "perf_stat": { 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ms": 0, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ms": 0, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ns": 0, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ns": 0 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.920 INFO:tasks.workunit.client.0.trial040.stdout: "alerts": [], 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "network_ping_times": [] 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "pool_stats": [ 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "poolid": 1, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg": 4, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "stat_sum": { 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes": 19, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects": 2, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_clones": 0, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_copies": 4, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing": 0, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_degraded": 2, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_misplaced": 0, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_unfound": 0, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_dirty": 2, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_whiteouts": 0, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_read": 0, 2026-02-13T07:04:18.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_read_kb": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_write": 2, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_write_kb": 2, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_scrub_errors": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_deep_scrub_errors": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_recovered": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_recovered": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_keys_recovered": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_omap": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_kb": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_kb": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_promote": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_high": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_low": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_some": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_full": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_pinned": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_legacy_snapsets": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_large_omap_objects": 0, 2026-02-13T07:04:18.922 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_manifest": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_bytes": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_keys": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_repaired": 0 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "store_stats": { 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "total": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "available": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 4096, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 19, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 0, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 0 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "log_size": 3, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "ondisk_log_size": 3, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "up": 4, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "acting": 4, 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: "num_store_stats": 1 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.923 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "osd_stats": [ 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "seq": 21474836482 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_by_state": [ 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "state": "active+undersized+degraded", 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "num": 2 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "state": "active+undersized", 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "num": 2 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "pool_pg_unavailable_map": [ 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "poolid": "1", 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "pgs": [] 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_by_osd": [ 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.924 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "num_primary_pg": 4, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "num_acting_pg": 4, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "num_up_not_acting_pg": 0 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "purged_snaps": [ 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "purged_snaps": {} 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "servicemap": { 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 1, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "modified": "2026-02-13T07:04:08.242392+0000", 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "services": {} 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "mgrstat_first_committed": 1, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "mgrstat_last_committed": 5, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "logm_first_committed": 1, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "logm_last_committed": 53, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "paxos": { 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "first_committed": 1, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "last_committed": 78, 2026-02-13T07:04:18.925 INFO:tasks.workunit.client.0.trial040.stdout: "last_pn": 0, 2026-02-13T07:04:18.926 INFO:tasks.workunit.client.0.trial040.stdout: "accepted_pn": 0 2026-02-13T07:04:18.926 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:18.926 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:report 327130429 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1690: test_wait_for_clean: run_osd td/ceph-helpers 1 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:18.928 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e182e4b6-5f69-4cf2-8a4c-568021ece6d3 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 e182e4b6-5f69-4cf2-8a4c-568021ece6d3' 2026-02-13T07:04:18.929 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 e182e4b6-5f69-4cf2-8a4c-568021ece6d3 2026-02-13T07:04:18.930 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:18.930 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDyzI5pQmfvNBAAFp22W44TiimRdHVd0NNCpg== 2026-02-13T07:04:18.930 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDyzI5pQmfvNBAAFp22W44TiimRdHVd0NNCpg=="}' 2026-02-13T07:04:18.930 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e182e4b6-5f69-4cf2-8a4c-568021ece6d3 -i td/ceph-helpers/1/new.json 2026-02-13T07:04:19.118 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:19.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T07:04:19.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDyzI5pQmfvNBAAFp22W44TiimRdHVd0NNCpg== --osd-uuid e182e4b6-5f69-4cf2-8a4c-568021ece6d3 2026-02-13T07:04:19.145 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:19.143+0000 7fa5733c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:19.146 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:19.144+0000 7fa5733c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:19.147 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:19.145+0000 7fa5733c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:19.148 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:19.146+0000 7fa5733c8900 -1 bdev(0x55f5f6f5cc00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:19.148 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:19.146+0000 7fa5733c8900 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T07:04:19.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T07:04:19.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:19.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:04:19.477 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:04:19.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:19.809 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:04:19.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:04:19.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:19.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:19.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:19.810 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:19.826 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:19.824+0000 7f2f74997900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:19.827 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:19.825+0000 7f2f74997900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:19.828 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:19.826+0000 7f2f74997900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:19.982 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:19.980+0000 7f2f74997900 -1 Falling back to public interface 2026-02-13T07:04:20.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:04:20.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:20.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:04:20.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:20.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:20.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:20.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:20.039 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:20.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:20.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:20.264 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:20.317 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:20.315+0000 7f2f74997900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:04:21.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:21.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:21.266 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:21.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:21.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:21.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:21.493 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 15 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3254051724,v1:127.0.0.1:6811/3254051724] [v2:127.0.0.1:6812/3254051724,v1:127.0.0.1:6813/3254051724] exists,up e182e4b6-5f69-4cf2-8a4c-568021ece6d3 2026-02-13T07:04:21.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:21.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:21.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:21.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1691: test_wait_for_clean: wait_for_clean 2026-02-13T07:04:21.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:04:21.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:04:21.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:04:21.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T07:04:21.496 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:04:21.496 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:04:21.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:04:21.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:04:21.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:04:21.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:04:21.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:04:21.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:04:21.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:04:21.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:04:21.578 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:04:21.814 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T07:04:21.814 INFO:tasks.workunit.client.0.trial040.stderr:1' 2026-02-13T07:04:21.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:04:21.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:04:21.815 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:04:21.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T07:04:21.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T07:04:21.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T07:04:21.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:04:21.880 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T07:04:21.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-13T07:04:21.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-13T07:04:21.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-64424509443' 2026-02-13T07:04:21.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:04:21.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T07:04:21.945 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:04:21.945 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:04:21.946 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-13T07:04:21.946 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:04:21.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-13T07:04:21.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T07:04:21.948 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836484 2026-02-13T07:04:21.948 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:04:22.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T07:04:22.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T07:04:23.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T07:04:23.178 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:04:23.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-13T07:04:23.414 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:04:23.415 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-64424509443 2026-02-13T07:04:23.415 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:04:23.416 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T07:04:23.416 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-64424509443 2026-02-13T07:04:23.417 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:04:23.418 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509443 2026-02-13T07:04:23.418 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 64424509443' 2026-02-13T07:04:23.418 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 64424509443 2026-02-13T07:04:23.418 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T07:04:23.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-13T07:04:23.647 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:04:23.647 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:23.647 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:23.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T07:04:23.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:23.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:23.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:23.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:23.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:23.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:23.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:24.188 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T07:04:24.189 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:24.189 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:24.189 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:24.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T07:04:24.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T07:04:24.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T07:04:24.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1692: test_wait_for_clean: teardown td/ceph-helpers 2026-02-13T07:04:24.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:24.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:24.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:24.507 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:24.507 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:24.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:24.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:24.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:24.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:24.613 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:24.613 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:24.614 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:24.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:24.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:24.616 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:24.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:24.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:24.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:24.617 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:24.618 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:24.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:24.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:24.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:24.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:24.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:24.624 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:24.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:24.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:24.626 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_wait_for_health_ok td/ceph-helpers 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1854: test_wait_for_health_ok: local dir=td/ceph-helpers 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1856: test_wait_for_health_ok: setup td/ceph-helpers 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:24.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:24.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:24.628 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:24.628 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:24.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:24.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:24.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:24.630 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:24.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:24.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:24.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:24.631 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:24.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:24.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:24.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:24.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:24.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:24.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:24.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:24.634 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:24.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:24.635 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:04:24.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:04:24.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:24.636 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:24.636 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:04:24.637 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:04:24.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:04:24.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:04:24.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:04:24.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1857: test_wait_for_health_ok: run_mon td/ceph-helpers a --osd_failsafe_full_ratio=.99 --mon_pg_warn_min_per_osd=0 2026-02-13T07:04:24.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:04:24.638 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:04:24.639 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:04:24.639 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:04:24.639 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:04:24.639 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_failsafe_full_ratio=.99 --mon_pg_warn_min_per_osd=0 2026-02-13T07:04:24.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:04:24.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:24.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:24.658 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:24.659 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:24.659 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:24.659 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:24.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_failsafe_full_ratio=.99 --mon_pg_warn_min_per_osd=0 2026-02-13T07:04:24.686 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:04:24.686 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:04:24.687 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:24.687 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:24.687 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:04:24.688 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:04:24.688 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:24.688 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:24.688 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:24.688 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:24.688 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:24.689 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:24.689 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:24.689 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:24.689 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:04:24.733 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:04:24.733 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:24.733 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:24.734 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:04:24.734 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:04:24.735 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:24.735 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:24.735 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:24.735 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:24.735 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:24.735 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:24.735 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:24.735 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:24.736 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:04:24.778 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1858: test_wait_for_health_ok: run_mgr td/ceph-helpers x --mon_pg_warn_min_per_osd=0 2026-02-13T07:04:24.778 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T07:04:24.778 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:04:24.778 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:04:24.778 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:04:24.778 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T07:04:24.778 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:04:24.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:04:24.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:24.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:24.882 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:24.882 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:24.882 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:24.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:24.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:04:24.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mon_pg_warn_min_per_osd=0 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1860: test_wait_for_health_ok: run_osd td/ceph-helpers 0 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:24.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:24.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:04:24.906 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:24.907 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=56cea0a2-d1a6-43c7-8ccb-ada719eb3d35 2026-02-13T07:04:24.907 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 56cea0a2-d1a6-43c7-8ccb-ada719eb3d35' 2026-02-13T07:04:24.908 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 56cea0a2-d1a6-43c7-8ccb-ada719eb3d35 2026-02-13T07:04:24.908 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:24.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD4zI5p8jJ8NhAAp9CLeUSwXcsoSIZiItqdqQ== 2026-02-13T07:04:24.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD4zI5p8jJ8NhAAp9CLeUSwXcsoSIZiItqdqQ=="}' 2026-02-13T07:04:24.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 56cea0a2-d1a6-43c7-8ccb-ada719eb3d35 -i td/ceph-helpers/0/new.json 2026-02-13T07:04:25.019 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:25.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T07:04:25.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD4zI5p8jJ8NhAAp9CLeUSwXcsoSIZiItqdqQ== --osd-uuid 56cea0a2-d1a6-43c7-8ccb-ada719eb3d35 2026-02-13T07:04:25.043 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:25.041+0000 7fec5e862900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:25.045 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:25.043+0000 7fec5e862900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:25.046 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:25.044+0000 7fec5e862900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:25.046 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:25.044+0000 7fec5e862900 -1 bdev(0x560fa02fec00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:25.047 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:25.044+0000 7fec5e862900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T07:04:25.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T07:04:25.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:25.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:04:25.470 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:04:25.471 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:25.577 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:04:25.578 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:04:25.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:25.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:25.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:25.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:25.594 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:25.592+0000 7fe597a97900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:25.596 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:25.594+0000 7fe597a97900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:25.597 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:25.595+0000 7fe597a97900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:25.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:04:25.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:25.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:04:25.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:25.677 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:25.677 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:25.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:25.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:25.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:25.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:25.777 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:25.802 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:25.800+0000 7fe597a97900 -1 Falling back to public interface 2026-02-13T07:04:26.191 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:26.188+0000 7fe597a97900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:04:26.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:26.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:26.779 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:26.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:26.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:26.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:26.877 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1875632290,v1:127.0.0.1:6801/1875632290] [v2:127.0.0.1:6802/1875632290,v1:127.0.0.1:6803/1875632290] exists,up 56cea0a2-d1a6-43c7-8ccb-ada719eb3d35 2026-02-13T07:04:26.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:26.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:26.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:26.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1861: test_wait_for_health_ok: run_osd td/ceph-helpers 1 2026-02-13T07:04:26.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:26.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:26.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:26.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:26.883 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:26.883 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:26.883 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:26.883 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:26.883 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:26.883 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:26.883 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:26.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:26.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:26.885 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T07:04:26.885 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:26.885 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e47f0bfc-3b4a-4291-be26-fa0c4bed956a 2026-02-13T07:04:26.885 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 e47f0bfc-3b4a-4291-be26-fa0c4bed956a' 2026-02-13T07:04:26.885 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 e47f0bfc-3b4a-4291-be26-fa0c4bed956a 2026-02-13T07:04:26.885 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:26.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD6zI5pLgYnNRAAe5RUVYGkxHHLO0c3joe+mQ== 2026-02-13T07:04:26.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD6zI5pLgYnNRAAe5RUVYGkxHHLO0c3joe+mQ=="}' 2026-02-13T07:04:26.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e47f0bfc-3b4a-4291-be26-fa0c4bed956a -i td/ceph-helpers/1/new.json 2026-02-13T07:04:26.999 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:27.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T07:04:27.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD6zI5pLgYnNRAAe5RUVYGkxHHLO0c3joe+mQ== --osd-uuid e47f0bfc-3b4a-4291-be26-fa0c4bed956a 2026-02-13T07:04:27.026 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:27.023+0000 7f0430c7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:27.027 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:27.024+0000 7f0430c7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:27.028 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:27.025+0000 7f0430c7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:27.028 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:27.025+0000 7f0430c7f900 -1 bdev(0x5600c1f6cc00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:27.029 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:27.025+0000 7f0430c7f900 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T07:04:27.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T07:04:27.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:27.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:04:27.350 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:04:27.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:27.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:04:27.669 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:04:27.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:27.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:27.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:27.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:27.685 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:27.682+0000 7f9299f52900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:27.687 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:27.684+0000 7f9299f52900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:27.688 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:27.685+0000 7f9299f52900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:27.822 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:27.818+0000 7f9299f52900 -1 Falling back to public interface 2026-02-13T07:04:27.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:04:27.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:27.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:04:27.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:27.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:27.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:27.895 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:27.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:27.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:27.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:28.121 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:28.167 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:28.164+0000 7f9299f52900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:04:29.122 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:29.122 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:29.122 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:29.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:29.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:29.123 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:29.347 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1709116702,v1:127.0.0.1:6811/1709116702] [v2:127.0.0.1:6812/1709116702,v1:127.0.0.1:6813/1709116702] exists,up e47f0bfc-3b4a-4291-be26-fa0c4bed956a 2026-02-13T07:04:29.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:29.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:29.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:29.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1862: test_wait_for_health_ok: run_osd td/ceph-helpers 2 2026-02-13T07:04:29.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:29.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:29.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T07:04:29.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/2 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/2' 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/2/journal' 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:29.352 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:29.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/2 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1a5f80ac-1a18-4894-b4b4-87e80a84c655 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 1a5f80ac-1a18-4894-b4b4-87e80a84c655' 2026-02-13T07:04:29.354 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 1a5f80ac-1a18-4894-b4b4-87e80a84c655 2026-02-13T07:04:29.355 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:29.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD9zI5p4CWTFRAADAnG5cp3UiutveAWsiBqYw== 2026-02-13T07:04:29.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD9zI5p4CWTFRAADAnG5cp3UiutveAWsiBqYw=="}' 2026-02-13T07:04:29.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1a5f80ac-1a18-4894-b4b4-87e80a84c655 -i td/ceph-helpers/2/new.json 2026-02-13T07:04:29.600 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:04:29.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/2/new.json 2026-02-13T07:04:29.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/2 --osd-journal=td/ceph-helpers/2/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD9zI5p4CWTFRAADAnG5cp3UiutveAWsiBqYw== --osd-uuid 1a5f80ac-1a18-4894-b4b4-87e80a84c655 2026-02-13T07:04:29.627 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:29.624+0000 7f7961bcd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:29.629 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:29.626+0000 7f7961bcd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:29.630 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:29.627+0000 7f7961bcd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:29.630 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:29.627+0000 7f7961bcd900 -1 bdev(0x56073b384c00 td/ceph-helpers/2/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:29.631 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:29.627+0000 7f7961bcd900 -1 bluestore(td/ceph-helpers/2) _read_fsid unparsable uuid 2026-02-13T07:04:29.944 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/2/keyring 2026-02-13T07:04:29.945 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:29.946 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T07:04:29.946 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T07:04:29.946 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:30.260 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:04:30.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T07:04:30.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/2 --osd-journal=td/ceph-helpers/2/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:30.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:30.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:30.261 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:30.277 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:30.274+0000 7efe31d05900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:30.279 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:30.276+0000 7efe31d05900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:30.280 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:30.277+0000 7efe31d05900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:30.413 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:30.410+0000 7efe31d05900 -1 Falling back to public interface 2026-02-13T07:04:30.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T07:04:30.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:30.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:04:30.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:30.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:30.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:30.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:30.485 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:30.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:30.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:04:30.712 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:30.721 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:30.718+0000 7efe31d05900 -1 osd.2 0 log_to_monitors true 2026-02-13T07:04:31.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:31.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:31.714 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:31.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:31.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:31.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:04:31.940 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3764268512,v1:127.0.0.1:6819/3764268512] [v2:127.0.0.1:6820/3764268512,v1:127.0.0.1:6821/3764268512] exists,up 1a5f80ac-1a18-4894-b4b4-87e80a84c655 2026-02-13T07:04:31.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:31.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:31.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:31.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1863: test_wait_for_health_ok: kill_daemons td/ceph-helpers TERM osd 2026-02-13T07:04:31.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:31.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:31.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:31.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:31.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:32.247 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:32.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1864: test_wait_for_health_ok: ceph osd down 0 2026-02-13T07:04:32.473 INFO:tasks.workunit.client.0.trial040.stderr:osd.0 is already down. 2026-02-13T07:04:32.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1866: test_wait_for_health_ok: TIMEOUT=1 2026-02-13T07:04:32.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1866: test_wait_for_health_ok: wait_for_health_ok 2026-02-13T07:04:32.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1850: wait_for_health_ok: wait_for_health HEALTH_OK 2026-02-13T07:04:32.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1829: wait_for_health: local grepstr=HEALTH_OK 2026-02-13T07:04:32.482 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1830: wait_for_health: get_timeout_delays 1 .1 2026-02-13T07:04:32.482 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:04:32.482 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:04:32.482 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:04:32.482 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:04:32.482 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:04:32.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1830: wait_for_health: delays=('0.1' '0.2' '0.4' '0.3') 2026-02-13T07:04:32.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1830: wait_for_health: local -a delays 2026-02-13T07:04:32.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1831: wait_for_health: local -i loop=0 2026-02-13T07:04:32.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: ceph health detail 2026-02-13T07:04:32.507 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: grep HEALTH_OK 2026-02-13T07:04:32.824 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1834: wait_for_health: (( 0 >= 4 )) 2026-02-13T07:04:32.824 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1838: wait_for_health: sleep 0.1 2026-02-13T07:04:32.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1839: wait_for_health: loop+=1 2026-02-13T07:04:32.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: ceph health detail 2026-02-13T07:04:32.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: grep HEALTH_OK 2026-02-13T07:04:33.254 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1834: wait_for_health: (( 1 >= 4 )) 2026-02-13T07:04:33.254 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1838: wait_for_health: sleep 0.2 2026-02-13T07:04:33.455 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1839: wait_for_health: loop+=1 2026-02-13T07:04:33.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: ceph health detail 2026-02-13T07:04:33.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: grep HEALTH_OK 2026-02-13T07:04:33.771 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1834: wait_for_health: (( 2 >= 4 )) 2026-02-13T07:04:33.771 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1838: wait_for_health: sleep 0.4 2026-02-13T07:04:34.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1839: wait_for_health: loop+=1 2026-02-13T07:04:34.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: ceph health detail 2026-02-13T07:04:34.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: grep HEALTH_OK 2026-02-13T07:04:34.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1834: wait_for_health: (( 3 >= 4 )) 2026-02-13T07:04:34.502 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1838: wait_for_health: sleep 0.3 2026-02-13T07:04:34.802 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1839: wait_for_health: loop+=1 2026-02-13T07:04:34.803 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: ceph health detail 2026-02-13T07:04:34.803 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: grep HEALTH_OK 2026-02-13T07:04:35.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1834: wait_for_health: (( 4 >= 4 )) 2026-02-13T07:04:35.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1835: wait_for_health: ceph health detail 2026-02-13T07:04:35.417 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_WARN 3 osds down; 1 host (3 osds) down; 1 root (3 osds) down 2026-02-13T07:04:35.417 INFO:tasks.workunit.client.0.trial040.stdout:[WRN] OSD_DOWN: 3 osds down 2026-02-13T07:04:35.417 INFO:tasks.workunit.client.0.trial040.stdout: osd.0 (root=default,host=trial040) is down 2026-02-13T07:04:35.417 INFO:tasks.workunit.client.0.trial040.stdout: osd.1 (root=default,host=trial040) is down 2026-02-13T07:04:35.418 INFO:tasks.workunit.client.0.trial040.stdout: osd.2 (root=default,host=trial040) is down 2026-02-13T07:04:35.418 INFO:tasks.workunit.client.0.trial040.stdout:[WRN] OSD_HOST_DOWN: 1 host (3 osds) down 2026-02-13T07:04:35.418 INFO:tasks.workunit.client.0.trial040.stdout: host trial040 (root=default) (3 osds) is down 2026-02-13T07:04:35.418 INFO:tasks.workunit.client.0.trial040.stdout:[WRN] OSD_ROOT_DOWN: 1 root (3 osds) down 2026-02-13T07:04:35.418 INFO:tasks.workunit.client.0.trial040.stdout: root default (3 osds) is down 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1836: wait_for_health: return 1 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1850: wait_for_health_ok: return 1 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1868: test_wait_for_health_ok: activate_osd td/ceph-helpers 0 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:35.428 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:35.429 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:04:35.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:04:35.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T07:04:35.431 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:04:35.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:35.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-13T07:04:35.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T07:04:35.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:04:35.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:04:35.432 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:04:35.445 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:35.442+0000 7f0c6f9b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:35.446 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:35.443+0000 7f0c6f9b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:35.447 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:35.444+0000 7f0c6f9b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:35.591 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:35.587+0000 7f0c6f9b0900 -1 Falling back to public interface 2026-02-13T07:04:35.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T07:04:35.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:35.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:04:35.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:35.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:35.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:35.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:35.656 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:35.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:35.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:35.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:35.941 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:35.938+0000 7f0c6f9b0900 -1 osd.0 14 log_to_monitors true 2026-02-13T07:04:36.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:36.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:36.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:36.883 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:36.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:36.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:37.107 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 18 up_thru 0 down_at 16 last_clean_interval [5,14) [v2:127.0.0.1:6800/1372918286,v1:127.0.0.1:6801/1372918286] [v2:127.0.0.1:6802/1372918286,v1:127.0.0.1:6803/1372918286] exists,up 56cea0a2-d1a6-43c7-8ccb-ada719eb3d35 2026-02-13T07:04:37.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:37.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:37.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:37.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1869: test_wait_for_health_ok: activate_osd td/ceph-helpers 1 2026-02-13T07:04:37.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T07:04:37.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:04:37.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/1 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:37.111 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:37.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:37.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:37.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:04:37.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:04:37.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/1 2026-02-13T07:04:37.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T07:04:37.113 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:04:37.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:37.114 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/1/whoami 2026-02-13T07:04:37.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T07:04:37.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:04:37.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:04:37.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:04:37.125 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:37.122+0000 7f707acf3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:37.126 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:37.123+0000 7f707acf3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:37.127 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:37.124+0000 7f707acf3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:37.266 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:37.263+0000 7f707acf3900 -1 Falling back to public interface 2026-02-13T07:04:37.339 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T07:04:37.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:37.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:04:37.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:37.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:37.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:37.340 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:37.340 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:37.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:37.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:37.560 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:37.557+0000 7f707acf3900 -1 osd.1 14 log_to_monitors true 2026-02-13T07:04:37.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:38.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:38.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:38.574 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:38.574 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:38.574 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:38.574 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:38.801 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 20 up_thru 0 down_at 16 last_clean_interval [10,14) [v2:127.0.0.1:6810/1102636097,v1:127.0.0.1:6811/1102636097] [v2:127.0.0.1:6812/1102636097,v1:127.0.0.1:6813/1102636097] exists,up e47f0bfc-3b4a-4291-be26-fa0c4bed956a 2026-02-13T07:04:38.804 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:38.804 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:38.804 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:38.804 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1870: test_wait_for_health_ok: activate_osd td/ceph-helpers 2 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/2 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/2' 2026-02-13T07:04:38.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/2/journal' 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:38.806 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:38.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:38.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:38.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:38.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:38.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:38.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:04:38.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:04:38.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/2 2026-02-13T07:04:38.807 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T07:04:38.808 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:04:38.808 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/2 --osd-journal=td/ceph-helpers/2/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:38.809 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/2/whoami 2026-02-13T07:04:38.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T07:04:38.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:04:38.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:04:38.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:04:38.820 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:38.817+0000 7f5b2196b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:38.821 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:38.819+0000 7f5b2196b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:38.822 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:38.819+0000 7f5b2196b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:38.961 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:38.957+0000 7f5b2196b900 -1 Falling back to public interface 2026-02-13T07:04:39.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T07:04:39.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:39.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:04:39.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:39.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:39.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:39.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:39.040 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:39.040 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:39.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:04:39.253 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:39.250+0000 7f5b2196b900 -1 osd.2 15 log_to_monitors true 2026-02-13T07:04:39.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:40.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:40.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:40.268 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:40.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:40.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:40.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:04:40.493 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 22 up_thru 0 down_at 16 last_clean_interval [0,0) [v2:127.0.0.1:6818/3811968978,v1:127.0.0.1:6819/3811968978] [v2:127.0.0.1:6820/3811968978,v1:127.0.0.1:6821/3811968978] exists,up 1a5f80ac-1a18-4894-b4b4-87e80a84c655 2026-02-13T07:04:40.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:40.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:40.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:40.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1871: test_wait_for_health_ok: wait_for_health_ok 2026-02-13T07:04:40.495 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1850: wait_for_health_ok: wait_for_health HEALTH_OK 2026-02-13T07:04:40.496 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1829: wait_for_health: local grepstr=HEALTH_OK 2026-02-13T07:04:40.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1830: wait_for_health: get_timeout_delays 300 .1 2026-02-13T07:04:40.496 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:04:40.496 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:04:40.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:04:40.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:04:40.496 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:04:40.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1830: wait_for_health: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '15' '15' '15' '15' '15' '15' '15' '15' '15' '15' '15' '15' '15' '15' '4.5') 2026-02-13T07:04:40.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1830: wait_for_health: local -a delays 2026-02-13T07:04:40.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1831: wait_for_health: local -i loop=0 2026-02-13T07:04:40.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: ceph health detail 2026-02-13T07:04:40.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1833: wait_for_health: grep HEALTH_OK 2026-02-13T07:04:40.983 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_OK 2026-02-13T07:04:40.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1872: test_wait_for_health_ok: teardown td/ceph-helpers 2026-02-13T07:04:40.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:40.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:40.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:40.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:40.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:40.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:40.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:40.986 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:41.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:41.092 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:41.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:41.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:41.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:41.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:41.094 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:41.095 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:41.096 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:41.096 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:41.096 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:41.097 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:41.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:41.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:41.105 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:41.105 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:41.105 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:41.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:41.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:41.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:41.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:04:41.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_wait_for_osd td/ceph-helpers 2026-02-13T07:04:41.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: test_wait_for_osd: local dir=td/ceph-helpers 2026-02-13T07:04:41.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: test_wait_for_osd: setup td/ceph-helpers 2026-02-13T07:04:41.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:04:41.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:04:41.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:41.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:41.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:41.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:41.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:41.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:41.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:41.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:41.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:41.109 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:41.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:41.109 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:41.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:41.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:41.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:41.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:41.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:41.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:41.112 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:41.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:41.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:41.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:41.114 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:41.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:41.115 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:41.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:41.115 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:41.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:41.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:04:41.117 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:04:41.117 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:41.117 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:41.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:04:41.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:04:41.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:04:41.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:04:41.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:04:41.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: test_wait_for_osd: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:04:41.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:04:41.119 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:04:41.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:04:41.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:04:41.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:04:41.120 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:04:41.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:04:41.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:41.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:41.140 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:41.141 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:41.141 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:41.141 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:41.141 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:04:41.167 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:04:41.167 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:04:41.168 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:41.168 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:41.168 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:04:41.168 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:04:41.169 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:41.169 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:41.169 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:41.169 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:41.169 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:41.170 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:41.170 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:41.170 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:41.170 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:04:41.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:04:41.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:41.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:41.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:04:41.215 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:04:41.216 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:41.216 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:41.216 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:41.217 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:41.217 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:41.217 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:41.217 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:41.217 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:41.217 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:04:41.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: test_wait_for_osd: run_mgr td/ceph-helpers x 2026-02-13T07:04:41.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T07:04:41.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:04:41.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:04:41.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:04:41.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T07:04:41.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:04:41.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:04:41.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:41.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:41.360 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:41.360 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:41.360 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:41.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:41.361 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:04:41.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:04:41.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:990: test_wait_for_osd: run_osd td/ceph-helpers 0 2026-02-13T07:04:41.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:41.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:41.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:04:41.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:41.383 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:41.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:41.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:41.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:41.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:41.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:41.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:41.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:04:41.385 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:41.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2f45a989-d1da-464c-9c70-2ca2cb339688 2026-02-13T07:04:41.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2f45a989-d1da-464c-9c70-2ca2cb339688' 2026-02-13T07:04:41.386 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 2f45a989-d1da-464c-9c70-2ca2cb339688 2026-02-13T07:04:41.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:41.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAJzY5pBx1dFxAAaOPBgIjma30OICKdrbS/4w== 2026-02-13T07:04:41.394 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAJzY5pBx1dFxAAaOPBgIjma30OICKdrbS/4w=="}' 2026-02-13T07:04:41.395 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2f45a989-d1da-464c-9c70-2ca2cb339688 -i td/ceph-helpers/0/new.json 2026-02-13T07:04:41.498 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:41.505 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T07:04:41.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAJzY5pBx1dFxAAaOPBgIjma30OICKdrbS/4w== --osd-uuid 2f45a989-d1da-464c-9c70-2ca2cb339688 2026-02-13T07:04:41.524 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:41.521+0000 7f56027dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:41.525 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:41.522+0000 7f56027dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:41.526 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:41.523+0000 7f56027dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:41.526 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:41.523+0000 7f56027dc900 -1 bdev(0x561287e44c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:41.527 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:41.523+0000 7f56027dc900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T07:04:41.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T07:04:41.948 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:41.949 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:04:41.949 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:04:41.949 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:42.055 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:04:42.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:04:42.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:42.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:42.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:42.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:42.072 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:42.069+0000 7fa7501a7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:42.074 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:42.071+0000 7fa7501a7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:42.075 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:42.072+0000 7fa7501a7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:42.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:04:42.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:42.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:04:42.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:42.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:42.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:42.159 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:42.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:42.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:42.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:42.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:42.323 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:42.320+0000 7fa7501a7900 -1 Falling back to public interface 2026-02-13T07:04:42.638 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:42.635+0000 7fa7501a7900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:04:43.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:43.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:43.259 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:43.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:43.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:43.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:43.358 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2460724967,v1:127.0.0.1:6801/2460724967] [v2:127.0.0.1:6802/2460724967,v1:127.0.0.1:6803/2460724967] exists,up 2f45a989-d1da-464c-9c70-2ca2cb339688 2026-02-13T07:04:43.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:43.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:43.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:43.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: test_wait_for_osd: run_osd td/ceph-helpers 1 2026-02-13T07:04:43.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:43.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:43.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:43.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:43.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5d6a3efd-f03c-4e31-a25f-6c99a418f0cb 2026-02-13T07:04:43.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 5d6a3efd-f03c-4e31-a25f-6c99a418f0cb' 2026-02-13T07:04:43.366 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 5d6a3efd-f03c-4e31-a25f-6c99a418f0cb 2026-02-13T07:04:43.366 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:43.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQALzY5pRuolFhAAUT3EeoyI+UKm1L8UTplTyw== 2026-02-13T07:04:43.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQALzY5pRuolFhAAUT3EeoyI+UKm1L8UTplTyw=="}' 2026-02-13T07:04:43.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5d6a3efd-f03c-4e31-a25f-6c99a418f0cb -i td/ceph-helpers/1/new.json 2026-02-13T07:04:43.478 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:43.484 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T07:04:43.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQALzY5pRuolFhAAUT3EeoyI+UKm1L8UTplTyw== --osd-uuid 5d6a3efd-f03c-4e31-a25f-6c99a418f0cb 2026-02-13T07:04:43.502 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:43.499+0000 7f641d564900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:43.503 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:43.500+0000 7f641d564900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:43.504 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:43.501+0000 7f641d564900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:43.505 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:43.502+0000 7f641d564900 -1 bdev(0x55bd1c4a8c00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:43.505 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:43.502+0000 7f641d564900 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T07:04:43.833 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T07:04:43.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:43.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:04:43.835 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:04:43.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:44.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:04:44.152 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:04:44.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:44.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:44.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:44.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:44.167 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:44.164+0000 7febbc212900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:44.169 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:44.166+0000 7febbc212900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:44.169 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:44.166+0000 7febbc212900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:44.323 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:44.320+0000 7febbc212900 -1 Falling back to public interface 2026-02-13T07:04:44.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:04:44.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:44.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:04:44.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:44.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:44.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:44.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:44.379 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:44.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:44.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:44.601 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:44.685 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:44.682+0000 7febbc212900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:04:45.603 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:45.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:45.604 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:45.605 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:45.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:45.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:45.829 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3439257877,v1:127.0.0.1:6811/3439257877] [v2:127.0.0.1:6812/3439257877,v1:127.0.0.1:6813/3439257877] exists,up 5d6a3efd-f03c-4e31-a25f-6c99a418f0cb 2026-02-13T07:04:45.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:45.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:45.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:45.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:992: test_wait_for_osd: wait_for_osd up 0 2026-02-13T07:04:45.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:45.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:04:45.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:45.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:45.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:45.831 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:45.832 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:45.832 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:45.832 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:46.057 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2460724967,v1:127.0.0.1:6801/2460724967] [v2:127.0.0.1:6802/2460724967,v1:127.0.0.1:6803/2460724967] exists,up 2f45a989-d1da-464c-9c70-2ca2cb339688 2026-02-13T07:04:46.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:46.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:46.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:46.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:993: test_wait_for_osd: wait_for_osd up 1 2026-02-13T07:04:46.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:46.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:04:46.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:46.059 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:46.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:46.060 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:46.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:46.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:46.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:04:46.287 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3439257877,v1:127.0.0.1:6811/3439257877] [v2:127.0.0.1:6812/3439257877,v1:127.0.0.1:6813/3439257877] exists,up 5d6a3efd-f03c-4e31-a25f-6c99a418f0cb 2026-02-13T07:04:46.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:46.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:46.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:46.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:994: test_wait_for_osd: kill_daemons td/ceph-helpers TERM osd.0 2026-02-13T07:04:46.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:46.289 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:46.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:46.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:46.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:46.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:46.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:995: test_wait_for_osd: wait_for_osd down 0 2026-02-13T07:04:46.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-13T07:04:46.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:04:46.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:46.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:46.595 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:46.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:46.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:46.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:46.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-13T07:04:46.823 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 11 last_clean_interval [0,0) [v2:127.0.0.1:6800/2460724967,v1:127.0.0.1:6801/2460724967] [v2:127.0.0.1:6802/2460724967,v1:127.0.0.1:6803/2460724967] exists 2f45a989-d1da-464c-9c70-2ca2cb339688 2026-02-13T07:04:46.824 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:996: test_wait_for_osd: TIMEOUT=1 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:996: test_wait_for_osd: wait_for_osd up 0 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 1 )) 2026-02-13T07:04:46.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:46.826 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:46.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:46.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:47.048 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:48.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:48.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 1 )) 2026-02-13T07:04:48.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 1 2026-02-13T07:04:48.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:997: test_wait_for_osd: teardown td/ceph-helpers 2026-02-13T07:04:48.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:48.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:48.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:48.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:48.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:48.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:48.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:48.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:48.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:48.159 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:48.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:48.160 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:48.161 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:48.162 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:48.162 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:48.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:48.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:48.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:48.163 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:48.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:48.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:48.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:48.169 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:48.169 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:48.170 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:48.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:48.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:48.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:48.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:04:48.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_wait_for_peered td/ceph-helpers 2026-02-13T07:04:48.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1766: test_wait_for_peered: local dir=td/ceph-helpers 2026-02-13T07:04:48.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1768: test_wait_for_peered: setup td/ceph-helpers 2026-02-13T07:04:48.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:04:48.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:04:48.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:04:48.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:04:48.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:04:48.173 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:04:48.173 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:04:48.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:04:48.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:04:48.173 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:04:48.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:04:48.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:04:48.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:04:48.175 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:04:48.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:04:48.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:04:48.176 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:04:48.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:48.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:04:48.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:04:48.177 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:04:48.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:04:48.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:04:48.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:04:48.180 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:04:48.180 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:48.180 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:48.180 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:04:48.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:04:48.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:04:48.181 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:04:48.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:04:48.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:48.183 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:48.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:04:48.184 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:04:48.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:04:48.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:04:48.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:04:48.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1769: test_wait_for_peered: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-13T07:04:48.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:04:48.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:04:48.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:04:48.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:04:48.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:04:48.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-13T07:04:48.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:04:48.205 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:48.206 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:48.206 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:48.206 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:48.206 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:48.206 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:48.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-13T07:04:48.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:04:48.233 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:04:48.233 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:48.233 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:48.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:04:48.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:04:48.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:48.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:48.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:48.235 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:48.235 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:48.235 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:48.235 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:48.235 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:48.236 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:04:48.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:04:48.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:04:48.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:04:48.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:04:48.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:04:48.280 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:04:48.280 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:04:48.280 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:04:48.281 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:04:48.281 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:48.281 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:48.282 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:04:48.282 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:04:48.282 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:04:48.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1770: test_wait_for_peered: run_osd td/ceph-helpers 0 2026-02-13T07:04:48.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:04:48.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:48.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:48.330 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=abf8513f-8cd5-43e2-a478-89637a68cf07 2026-02-13T07:04:48.331 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 abf8513f-8cd5-43e2-a478-89637a68cf07 2026-02-13T07:04:48.331 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 abf8513f-8cd5-43e2-a478-89637a68cf07' 2026-02-13T07:04:48.331 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:48.339 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAQzY5pqe8XFBAA7Dy8m9HjsKe1xdfS5t95KQ== 2026-02-13T07:04:48.339 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAQzY5pqe8XFBAA7Dy8m9HjsKe1xdfS5t95KQ=="}' 2026-02-13T07:04:48.339 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new abf8513f-8cd5-43e2-a478-89637a68cf07 -i td/ceph-helpers/0/new.json 2026-02-13T07:04:48.444 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:48.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T07:04:48.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAQzY5pqe8XFBAA7Dy8m9HjsKe1xdfS5t95KQ== --osd-uuid abf8513f-8cd5-43e2-a478-89637a68cf07 2026-02-13T07:04:48.469 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:48.466+0000 7fa5f17c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:48.470 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:48.467+0000 7fa5f17c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:48.471 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:48.468+0000 7fa5f17c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:48.472 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:48.469+0000 7fa5f17c6900 -1 bdev(0x557668b1cc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:48.472 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:48.469+0000 7fa5f17c6900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T07:04:48.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T07:04:48.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:48.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:04:48.792 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:04:48.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:48.895 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:04:48.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:04:48.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:48.896 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:48.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:48.897 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:48.911 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:48.908+0000 7fd58cbb0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:48.913 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:48.910+0000 7fd58cbb0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:48.913 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:48.910+0000 7fd58cbb0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:48.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:04:48.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:04:48.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:04:48.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:04:48.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:04:48.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:48.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:04:48.996 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:04:48.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:48.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:49.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:04:49.115 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:49.112+0000 7fd58cbb0900 -1 Falling back to public interface 2026-02-13T07:04:49.361 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:49.358+0000 7fd58cbb0900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:04:50.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:04:50.093 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:04:50.093 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:50.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:04:50.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:04:50.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:04:50.192 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1221926425,v1:127.0.0.1:6801/1221926425] [v2:127.0.0.1:6802/1221926425,v1:127.0.0.1:6803/1221926425] exists,up abf8513f-8cd5-43e2-a478-89637a68cf07 2026-02-13T07:04:50.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:04:50.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:04:50.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:04:50.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1771: test_wait_for_peered: run_mgr td/ceph-helpers x 2026-02-13T07:04:50.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T07:04:50.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:04:50.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:04:50.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:04:50.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T07:04:50.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:04:50.294 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:04:50.294 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:50.294 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:50.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:50.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:50.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:50.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:50.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:04:50.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:04:50.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1772: test_wait_for_peered: create_rbd_pool 2026-02-13T07:04:50.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T07:04:50.408 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T07:04:50.415 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T07:04:50.415 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T07:04:50.592 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T07:04:50.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T07:04:51.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T07:04:51.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1773: test_wait_for_peered: WAIT_FOR_CLEAN_TIMEOUT=1 2026-02-13T07:04:51.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1773: test_wait_for_peered: wait_for_clean 2026-02-13T07:04:51.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:04:51.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:04:51.880 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:04:51.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 1 .1 2026-02-13T07:04:51.881 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:04:51.882 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:04:51.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:04:51.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:04:51.882 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:04:51.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.3') 2026-02-13T07:04:51.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:04:51.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:04:51.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:04:51.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:04:51.905 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:04:52.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T07:04:52.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:04:52.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:04:52.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:04:52.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=0 2026-02-13T07:04:52.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 0 2026-02-13T07:04:52.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-0' 2026-02-13T07:04:52.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:04:52.072 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-0 2026-02-13T07:04:52.072 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:04:52.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:04:52.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-0 2026-02-13T07:04:52.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:04:52.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=0 2026-02-13T07:04:52.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 0' 2026-02-13T07:04:52.074 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 0 2026-02-13T07:04:52.074 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:04:52.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 0 -lt 0 2026-02-13T07:04:52.176 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:04:52.177 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:52.177 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:52.277 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 0 == 0 2026-02-13T07:04:52.277 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1655: wait_for_clean: sleep 1 2026-02-13T07:04:53.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:04:53.279 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:53.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:53.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T07:04:53.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:53.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:53.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:53.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:53.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:53.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:53.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:53.812 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:53.813 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:53.813 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:53.813 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:54.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:54.125 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-13T07:04:54.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-13T07:04:54.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-13T07:04:54.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T07:04:54.126 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-13T07:04:54.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T07:04:54.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:54.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:54.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:54.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:54.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:54.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:54.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:54.455 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:54.455 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:54.456 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:54.456 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:54.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:54.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-13T07:04:54.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-13T07:04:54.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T07:04:54.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T07:04:54.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T07:04:54.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T07:04:54.775 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T07:04:54.775 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T07:04:55.090 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T07:04:55.090 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T07:04:55.090 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 4 )) 2026-02-13T07:04:55.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T07:04:55.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-13T07:04:55.291 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T07:04:55.291 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:55.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:55.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:55.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:55.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:55.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:55.292 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:55.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:55.522 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:55.523 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:55.523 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:55.833 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:55.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-13T07:04:55.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-13T07:04:55.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T07:04:55.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T07:04:55.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T07:04:55.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T07:04:55.834 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T07:04:55.834 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T07:04:56.147 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T07:04:56.147 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T07:04:56.147 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 4 )) 2026-02-13T07:04:56.147 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T07:04:56.147 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-13T07:04:56.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T07:04:56.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:56.549 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:56.549 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:56.549 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:56.549 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:56.549 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:56.550 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:56.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:56.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:56.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:56.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:57.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:57.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-13T07:04:57.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-13T07:04:57.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T07:04:57.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T07:04:57.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T07:04:57.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T07:04:57.087 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T07:04:57.087 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T07:04:57.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T07:04:57.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T07:04:57.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 4 )) 2026-02-13T07:04:57.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T07:04:57.397 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.3 2026-02-13T07:04:57.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T07:04:57.698 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:04:57.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:04:57.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:04:57.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:04:57.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:04:57.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:04:57.699 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:04:57.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-13T07:04:57.922 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:04:57.922 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:04:57.922 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:04:58.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 4 2026-02-13T07:04:58.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-13T07:04:58.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-13T07:04:58.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T07:04:58.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T07:04:58.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T07:04:58.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T07:04:58.236 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T07:04:58.236 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T07:04:58.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T07:04:58.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T07:04:58.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 4 )) 2026-02-13T07:04:58.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-13T07:04:58.863 INFO:tasks.workunit.client.0.trial040.stdout:{ 2026-02-13T07:04:58.863 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_fingerprint": "2d11b62b-1ee2-4ced-a354-7a072b17870e", 2026-02-13T07:04:58.863 INFO:tasks.workunit.client.0.trial040.stdout: "version": "20.2.0-636-g3b40640f", 2026-02-13T07:04:58.863 INFO:tasks.workunit.client.0.trial040.stdout: "commit": "3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb", 2026-02-13T07:04:58.863 INFO:tasks.workunit.client.0.trial040.stdout: "timestamp": "2026-02-13T07:04:58.860925+0000", 2026-02-13T07:04:58.863 INFO:tasks.workunit.client.0.trial040.stdout: "tag": "", 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "health": { 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "status": "HEALTH_WARN", 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "checks": { 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "PG_DEGRADED": { 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "message": "Degraded data redundancy: 2/4 objects degraded (50.000%), 2 pgs degraded", 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "count": 2 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [ 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.864 INFO:tasks.workunit.client.0.trial040.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0]" 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0]" 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: "TOO_FEW_OSDS": { 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: "severity": "HEALTH_WARN", 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: "summary": { 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: "message": "OSD count 1 < osd_pool_default_size 3", 2026-02-13T07:04:58.865 INFO:tasks.workunit.client.0.trial040.stdout: "count": 2 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: "detail": [], 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: "muted": false 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: "mutes": [] 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: "monmap_first_committed": 1, 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: "monmap_last_committed": 1, 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: "monmap": { 2026-02-13T07:04:58.866 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 1, 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "fsid": "1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2", 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "modified": "2026-02-13T07:04:48.197420Z", 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "created": "2026-02-13T07:04:48.197420Z", 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "min_mon_release": 20, 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "min_mon_release_name": "tentacle", 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "election_strategy": 1, 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "disallowed_leaders": "", 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode": false, 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "tiebreaker_mon": "", 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "removed_ranks": "", 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "features": { 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "persistent": [ 2026-02-13T07:04:58.867 INFO:tasks.workunit.client.0.trial040.stdout: "kraken", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "luminous", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "mimic", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap-prune", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "nautilus", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "octopus", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "pacific", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "elector-pinging", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "quincy", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "reef", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "squid", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "tentacle", 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: "nvmeof_beacon_diff" 2026-02-13T07:04:58.868 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: "optional": [] 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: "mons": [ 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: "rank": 0, 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: "name": "a", 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: "public_addrs": { 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:7109", 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 0 2026-02-13T07:04:58.869 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:7109/0", 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: "public_addr": "127.0.0.1:7109/0", 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: "priority": 0, 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 0, 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: "crush_location": "{}" 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: "quorum": [ 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: 0 2026-02-13T07:04:58.870 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap": { 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 12, 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "fsid": "1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2", 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "created": "2026-02-13T07:04:48.233340+0000", 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "modified": "2026-02-13T07:04:51.872563+0000", 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "last_up_change": "2026-02-13T07:04:49.968237+0000", 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "last_in_change": "2026-02-13T07:04:48.420057+0000", 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "flags_num": 5799936, 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "flags_set": [ 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "pglog_hardlimit", 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "purged_snapdirs", 2026-02-13T07:04:58.871 INFO:tasks.workunit.client.0.trial040.stdout: "recovery_deletes", 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "sortbitwise" 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "crush_version": 3, 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "full_ratio": 0.99000000953674316, 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_snapshot": "", 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "pool_max": 1, 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "max_osd": 1, 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "require_min_compat_client": "luminous", 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "min_compat_client": "jewel", 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "require_osd_release": "tentacle", 2026-02-13T07:04:58.872 INFO:tasks.workunit.client.0.trial040.stdout: "allow_crimson": false, 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "pools": [ 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "pool_name": "rbd", 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "create_time": "2026-02-13T07:04:50.494119+0000", 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "flags": 8193, 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "type": 1, 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "size": 2, 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "min_size": 1, 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "crush_rule": 0, 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_count": 0, 2026-02-13T07:04:58.873 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_target": 0, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_barrier": 0, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "is_stretch_pool": false, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "object_hash": 2, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "pg_autoscale_mode": "off", 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "pg_num": 4, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "pg_placement_num": 4, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "pg_placement_num_target": 4, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "pg_num_target": 4, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "pg_num_pending": 4, 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "last_pg_merge_meta": { 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "source_pgid": "0.0", 2026-02-13T07:04:58.874 INFO:tasks.workunit.client.0.trial040.stdout: "ready_epoch": 0, 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "last_epoch_started": 0, 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "last_epoch_clean": 0, 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "source_version": "0'0", 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "target_version": "0'0" 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "last_change": "12", 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "last_force_op_resend": "0", 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "last_force_op_resend_preluminous": "0", 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "auid": 0, 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "snap_mode": "selfmanaged", 2026-02-13T07:04:58.875 INFO:tasks.workunit.client.0.trial040.stdout: "snap_seq": 2, 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "snap_epoch": 12, 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "pool_snaps": [], 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "removed_snaps": "[]", 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "quota_max_bytes": 0, 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "quota_max_objects": 0, 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "tiers": [], 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "tier_of": -1, 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "read_tier": -1, 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "write_tier": -1, 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "cache_mode": "none", 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "target_max_bytes": 0, 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "target_max_objects": 0, 2026-02-13T07:04:58.876 INFO:tasks.workunit.client.0.trial040.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "cache_min_flush_age": 0, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "cache_min_evict_age": 0, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "erasure_code_profile": "", 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_params": { 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "type": "none" 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_period": 0, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_count": 0, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "use_gmt_hitset": true, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "min_read_recency_for_promote": 0, 2026-02-13T07:04:58.877 INFO:tasks.workunit.client.0.trial040.stdout: "min_write_recency_for_promote": 0, 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_grade_decay_rate": 0, 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "hit_set_search_last_n": 0, 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "grade_table": [], 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "stripe_width": 0, 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "expected_num_objects": 0, 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "fast_read": false, 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "nonprimary_shards": "{}", 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "options": {}, 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "application_metadata": { 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "rbd": {} 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "read_balance": { 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "score_type": "Fair distribution", 2026-02-13T07:04:58.878 INFO:tasks.workunit.client.0.trial040.stdout: "score_acting": 1, 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: "score_stable": 1, 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: "optimal_score": 1, 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: "raw_score_acting": 1, 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: "raw_score_stable": 1, 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: "primary_affinity_weighted": 1, 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: "average_primary_affinity": 1, 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: "average_primary_affinity_weighted": 1 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: "osds": [ 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.879 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "uuid": "abf8513f-8cd5-43e2-a478-89637a68cf07", 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "up": 1, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "in": 1, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 1, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "primary_affinity": 1, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "last_clean_begin": 0, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "last_clean_end": 0, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "up_from": 5, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "up_thru": 7, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "down_at": 0, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "lost_at": 0, 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "public_addrs": { 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:58.880 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6800", 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1221926425 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6801", 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1221926425 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.881 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_addrs": { 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6802", 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1221926425 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6803", 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1221926425 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.882 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_back_addrs": { 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6806", 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1221926425 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6807", 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1221926425 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.883 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_front_addrs": { 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: "addrvec": [ 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v2", 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6804", 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1221926425 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: "type": "v1", 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: "addr": "127.0.0.1:6805", 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: "nonce": 1221926425 2026-02-13T07:04:58.884 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: "public_addr": "127.0.0.1:6801/1221926425", 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: "cluster_addr": "127.0.0.1:6803/1221926425", 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_back_addr": "127.0.0.1:6807/1221926425", 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: "heartbeat_front_addr": "127.0.0.1:6805/1221926425", 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: "state": [ 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: "exists", 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: "up" 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.885 INFO:tasks.workunit.client.0.trial040.stdout: "osd_xinfo": [ 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "down_stamp": "0.000000", 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "laggy_probability": 0, 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "laggy_interval": 0, 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "features": 4544132024016699391, 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "old_weight": 0, 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "last_purged_snaps_scrub": "2026-02-13T07:04:50.319652+0000", 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "dead_epoch": 0 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "pg_upmap": [], 2026-02-13T07:04:58.886 INFO:tasks.workunit.client.0.trial040.stdout: "pg_upmap_items": [], 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "pg_upmap_primaries": [], 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "pg_temp": [], 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "primary_temp": [], 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "blocklist": {}, 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "range_blocklist": {}, 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "erasure_code_profiles": { 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "default": { 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "k": "2", 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "m": "2", 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "plugin": "isa", 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: "technique": "reed_sol_van" 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.887 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: "removed_snaps_queue": [ 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: "snaps": [ 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: "begin": 2, 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: "length": 1 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: "new_removed_snaps": [ 2026-02-13T07:04:58.888 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: "snaps": [ 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: "begin": 2, 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: "length": 1 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: "new_purged_snaps": [], 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: "crush_node_flags": {}, 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: "device_class_flags": {}, 2026-02-13T07:04:58.889 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode": { 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode_enabled": false, 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_bucket_count": 0, 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "degraded_stretch_mode": 0, 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "recovering_stretch_mode": 0, 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "stretch_mode_bucket": 0 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "osd_metadata": [ 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "arch": "x86_64", 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "back_addr": "[v2:127.0.0.1:6802/1221926425,v1:127.0.0.1:6803/1221926425]", 2026-02-13T07:04:58.890 INFO:tasks.workunit.client.0.trial040.stdout: "back_iface": "", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs": "1", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs_dedicated_db": "0", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs_dedicated_wal": "0", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluefs_single_shared_device": "1", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_allocation_from_file": "1", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_allocator": "hybrid", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_access_mode": "file", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_block_size": "4096", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-13T07:04:58.891 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_path": "td/ceph-helpers/0/block", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_rotational": "0", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_size": "107374182400", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_support_discard": "1", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_bdev_type": "ssd", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_min_alloc_size": "4096", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "bluestore_write_mode": "classic", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_release": "tentacle", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_version": "ceph version 20.2.0-636-g3b40640f (3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb) tentacle (stable - RelWithDebInfo)", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_version_short": "20.2.0-636-g3b40640f", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "ceph_version_when_created": "ceph version 20.2.0-636-g3b40640f (3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb) tentacle (stable - RelWithDebInfo)", 2026-02-13T07:04:58.892 INFO:tasks.workunit.client.0.trial040.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "created_at": "2026-02-13T07:04:48.745265Z", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "default_device_class": "ssd", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6793", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "devices": "nvme0n1", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "distro": "centos", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "distro_description": "CentOS Stream 9", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "distro_version": "9", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "front_addr": "[v2:127.0.0.1:6800/1221926425,v1:127.0.0.1:6801/1221926425]", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "front_iface": "", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/1221926425,v1:127.0.0.1:6807/1221926425]", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/1221926425,v1:127.0.0.1:6805/1221926425]", 2026-02-13T07:04:58.893 INFO:tasks.workunit.client.0.trial040.stdout: "hostname": "trial040", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "journal_rotational": "0", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 6 13:57:07 UTC 2026", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "kernel_version": "5.14.0-677.el9.x86_64", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "mem_swap_kb": "8388604", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "mem_total_kb": "130438944", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "objectstore_numa_node": "0", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "objectstore_numa_nodes": "0", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "os": "Linux", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "osd_data": "td/ceph-helpers/0", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "osd_objectstore": "bluestore", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "osdspec_affinity": "", 2026-02-13T07:04:58.894 INFO:tasks.workunit.client.0.trial040.stdout: "rotational": "0" 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap_clean_epochs": { 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: "min_last_epoch_clean": 0, 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: "last_epoch_clean": { 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: "per_pool": [] 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: "osd_epochs": [ 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 5 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.895 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap_first_committed": 1, 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: "osdmap_last_committed": 12, 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: "crushmap": { 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: "devices": [ 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: "name": "osd.0", 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: "class": "ssd" 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: "types": [ 2026-02-13T07:04:58.896 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 0, 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: "name": "osd" 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 1, 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: "name": "host" 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 2, 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: "name": "chassis" 2026-02-13T07:04:58.897 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 3, 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: "name": "rack" 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 4, 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: "name": "row" 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 5, 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: "name": "pdu" 2026-02-13T07:04:58.898 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 6, 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: "name": "pod" 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 7, 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: "name": "room" 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 8, 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: "name": "datacenter" 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.899 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 9, 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: "name": "zone" 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 10, 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: "name": "region" 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 11, 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: "name": "root" 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: "buckets": [ 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.900 INFO:tasks.workunit.client.0.trial040.stdout: "id": -1, 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "name": "default", 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 11, 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "root", 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "id": -3, 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.901 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "id": -2, 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "name": "default~ssd", 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 11, 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "root", 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "id": -4, 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.902 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "id": -3, 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "name": "trial040", 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 1, 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "host", 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.903 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "id": -4, 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "name": "trial040~ssd", 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "type_id": 1, 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "type_name": "host", 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "alg": "straw2", 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "hash": "rjenkins1", 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "items": [ 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "id": 0, 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "weight": 6402, 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "pos": 0 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.904 INFO:tasks.workunit.client.0.trial040.stdout: "rules": [ 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "rule_id": 0, 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "rule_name": "replicated_rule", 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "type": 1, 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "steps": [ 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "op": "take", 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "item": -1, 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "item_name": "default" 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "op": "choose_firstn", 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "num": 0, 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "type": "osd" 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.905 INFO:tasks.workunit.client.0.trial040.stdout: "op": "emit" 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: ] 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "tunables": { 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "choose_local_tries": 0, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "choose_local_fallback_tries": 0, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "choose_total_tries": 50, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "chooseleaf_descend_once": 1, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "chooseleaf_vary_r": 1, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "chooseleaf_stable": 1, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "msr_descents": 100, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "msr_collision_tries": 100, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "straw_calc_version": 1, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "allowed_bucket_algs": 54, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "profile": "jewel", 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "optimal_tunables": 1, 2026-02-13T07:04:58.906 INFO:tasks.workunit.client.0.trial040.stdout: "legacy_tunables": 0, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "minimum_required_version": "jewel", 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables": 1, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables2": 1, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "has_v2_rules": 0, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables3": 1, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "has_v3_rules": 0, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "has_v4_buckets": 1, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "require_feature_tunables5": 1, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "has_v5_rules": 0, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "has_msr_rules": 0 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "choose_args": {} 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "fsmap": { 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 1, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "btime": "2026-02-13T07:04:48:233195+0000", 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "default_fscid": -1, 2026-02-13T07:04:58.907 INFO:tasks.workunit.client.0.trial040.stdout: "compat": { 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "compat": {}, 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "ro_compat": {}, 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "incompat": { 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_1": "base v0.20", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_2": "client writeable ranges", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_3": "default file layouts on dirs", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_4": "dir inode in separate object", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_5": "mds uses versioned encoding", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_8": "no anchor table", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_9": "file layout v2", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_10": "snaprealm v2", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_11": "minor log segments", 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_12": "quiesce subvolumes" 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "feature_flags": { 2026-02-13T07:04:58.908 INFO:tasks.workunit.client.0.trial040.stdout: "enable_multiple": true, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "ever_enabled_multiple": true 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "standbys": [], 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "filesystems": [] 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "mdsmap_first_committed": 1, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "mdsmap_last_committed": 1, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "auth": { 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "first_committed": 1, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "last_committed": 3, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "num_secrets": 1 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg": 4, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_active": 4, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_unknown": 0, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "num_osd": 1, 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "pool_sum": { 2026-02-13T07:04:58.909 INFO:tasks.workunit.client.0.trial040.stdout: "stat_sum": { 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes": 19, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects": 2, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_clones": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_copies": 4, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_degraded": 2, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_misplaced": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_unfound": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_dirty": 2, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_whiteouts": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_read": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_read_kb": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_write": 2, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_write_kb": 2, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_scrub_errors": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T07:04:58.910 INFO:tasks.workunit.client.0.trial040.stdout: "num_deep_scrub_errors": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_recovered": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_recovered": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_keys_recovered": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_omap": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_kb": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_kb": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_promote": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_high": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_low": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_some": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_full": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_pinned": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_legacy_snapsets": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_large_omap_objects": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_manifest": 0, 2026-02-13T07:04:58.911 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_bytes": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_keys": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_repaired": 0 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "store_stats": { 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "total": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "available": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 0, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 0 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "log_size": 3, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "ondisk_log_size": 3, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "up": 4, 2026-02-13T07:04:58.912 INFO:tasks.workunit.client.0.trial040.stdout: "acting": 4, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "num_store_stats": 0 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "osd_sum": { 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "up_from": 0, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "seq": 0, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "num_pgs": 4, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "num_osds": 1, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_osds": 1, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_omap_osds": 1, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "kb": 104857600, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used": 27068, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_data": 108, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_omap": 5, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_meta": 26938, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "kb_avail": 104830532, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "statfs": { 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "total": 107374182400, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "available": 107346464768, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:04:58.913 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 110592, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 22886, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 5530, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 27585126 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "hb_peers": [], 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "snap_trim_queue_len": 0, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_snap_trimming": 0, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "num_shards_repaired": 0, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "op_queue_age_hist": { 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "histogram": [], 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "upper_bound": 1 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "perf_stat": { 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ms": 0, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ms": 0, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ns": 0, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ns": 0 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.914 INFO:tasks.workunit.client.0.trial040.stdout: "alerts": [], 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "network_ping_times": [] 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "osd_sum_by_class": { 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "ssd": { 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "up_from": 0, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "seq": 0, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_pgs": 4, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_osds": 1, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_osds": 1, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "num_per_pool_omap_osds": 1, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "kb": 104857600, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used": 27068, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_data": 108, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_omap": 5, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "kb_used_meta": 26938, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "kb_avail": 104830532, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "statfs": { 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "total": 107374182400, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "available": 107346464768, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:04:58.915 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 110592, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 22886, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 5530, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 27585126 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "hb_peers": [], 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "snap_trim_queue_len": 0, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "num_snap_trimming": 0, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "num_shards_repaired": 0, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "op_queue_age_hist": { 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "histogram": [], 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "upper_bound": 1 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "perf_stat": { 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ms": 0, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ms": 0, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "commit_latency_ns": 0, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "apply_latency_ns": 0 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.916 INFO:tasks.workunit.client.0.trial040.stdout: "alerts": [], 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "network_ping_times": [] 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "pool_stats": [ 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "poolid": 1, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg": 4, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "stat_sum": { 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes": 19, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects": 2, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_clones": 0, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_object_copies": 4, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_missing": 0, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_degraded": 2, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_misplaced": 0, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_unfound": 0, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_dirty": 2, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_whiteouts": 0, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_read": 0, 2026-02-13T07:04:58.917 INFO:tasks.workunit.client.0.trial040.stdout: "num_read_kb": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_write": 2, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_write_kb": 2, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_scrub_errors": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_deep_scrub_errors": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_recovered": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_recovered": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_keys_recovered": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_omap": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_kb": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_kb": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_promote": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_high": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_flush_mode_low": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_some": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_evict_mode_full": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_pinned": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_legacy_snapsets": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_large_omap_objects": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_manifest": 0, 2026-02-13T07:04:58.918 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_bytes": 0, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "num_omap_keys": 0, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "num_objects_repaired": 0 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "store_stats": { 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "total": 0, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "available": 0, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "internally_reserved": 0, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "allocated": 4096, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "data_stored": 19, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed": 0, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_allocated": 0, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "data_compressed_original": 0, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "omap_allocated": 0, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "internal_metadata": 0 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "log_size": 3, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "ondisk_log_size": 3, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "up": 4, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "acting": 4, 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "num_store_stats": 1 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.919 INFO:tasks.workunit.client.0.trial040.stdout: "osd_stats": [ 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "seq": 21474836482 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_by_state": [ 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "state": "active+undersized+degraded", 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "num": 2 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "state": "active+undersized", 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "num": 2 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "pool_pg_unavailable_map": [ 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "poolid": "1", 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "pgs": [] 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "num_pg_by_osd": [ 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "osd": 0, 2026-02-13T07:04:58.920 INFO:tasks.workunit.client.0.trial040.stdout: "num_primary_pg": 4, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_acting_pg": 4, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "num_up_not_acting_pg": 0 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "purged_snaps": [ 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: { 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "pool": 1, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "purged_snaps": {} 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: ], 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "servicemap": { 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "epoch": 1, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "modified": "2026-02-13T07:04:48.233460+0000", 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "services": {} 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: }, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "mgrstat_first_committed": 1, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "mgrstat_last_committed": 5, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "logm_first_committed": 1, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "logm_last_committed": 54, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "paxos": { 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "first_committed": 1, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "last_committed": 80, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "last_pn": 0, 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: "accepted_pn": 0 2026-02-13T07:04:58.921 INFO:tasks.workunit.client.0.trial040.stdout: } 2026-02-13T07:04:58.922 INFO:tasks.workunit.client.0.trial040.stdout:} 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:report 853906060 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1774: test_wait_for_peered: run_osd td/ceph-helpers 1 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:04:58.924 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=82b3dc13-9423-4b68-81f9-7d214e015acd 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 82b3dc13-9423-4b68-81f9-7d214e015acd' 2026-02-13T07:04:58.925 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 82b3dc13-9423-4b68-81f9-7d214e015acd 2026-02-13T07:04:58.926 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:04:58.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAazY5pLrO6NBAANf+WWwu+/V42K7TitPZ1DA== 2026-02-13T07:04:58.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAazY5pLrO6NBAANf+WWwu+/V42K7TitPZ1DA=="}' 2026-02-13T07:04:58.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 82b3dc13-9423-4b68-81f9-7d214e015acd -i td/ceph-helpers/1/new.json 2026-02-13T07:04:59.113 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:04:59.121 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T07:04:59.122 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAazY5pLrO6NBAANf+WWwu+/V42K7TitPZ1DA== --osd-uuid 82b3dc13-9423-4b68-81f9-7d214e015acd 2026-02-13T07:04:59.140 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:59.138+0000 7f1f189dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:59.141 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:59.139+0000 7f1f189dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:59.142 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:59.140+0000 7f1f189dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:59.143 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:59.141+0000 7f1f189dc900 -1 bdev(0x557e6e9ccc00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:04:59.143 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:59.141+0000 7f1f189dc900 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T07:04:59.459 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T07:04:59.459 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:04:59.460 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:04:59.461 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:04:59.461 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:04:59.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:04:59.773 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:04:59.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:04:59.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:04:59.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:04:59.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:04:59.789 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:59.787+0000 7fd18f72d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:59.791 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:59.789+0000 7fd18f72d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:59.792 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:59.790+0000 7fd18f72d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:04:59.918 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:04:59.916+0000 7fd18f72d900 -1 Falling back to public interface 2026-02-13T07:05:00.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:05:00.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:05:00.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:05:00.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:05:00.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:05:00.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:00.001 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:00.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:05:00.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:00.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:05:00.211 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:00.209+0000 7fd18f72d900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:05:00.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:01.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:01.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:01.231 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:01.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:05:01.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:01.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:05:01.456 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1400105571,v1:127.0.0.1:6811/1400105571] [v2:127.0.0.1:6812/1400105571,v1:127.0.0.1:6813/1400105571] exists,up 82b3dc13-9423-4b68-81f9-7d214e015acd 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1775: test_wait_for_peered: wait_for_peered 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1731: wait_for_peered: local cmd= 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1732: wait_for_peered: local num_peered=-1 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1733: wait_for_peered: local cur_peered 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: get_timeout_delays 90 .1 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:05:01.457 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:05:01.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:05:01.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: local -a delays 2026-02-13T07:05:01.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1735: wait_for_peered: local -i loop=0 2026-02-13T07:05:01.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1737: wait_for_peered: flush_pg_stats 2026-02-13T07:05:01.539 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:05:01.540 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:05:01.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T07:05:01.773 INFO:tasks.workunit.client.0.trial040.stderr:1' 2026-02-13T07:05:01.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:05:01.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:01.773 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:05:01.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T07:05:01.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T07:05:01.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T07:05:01.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:01.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T07:05:01.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=68719476739 2026-02-13T07:05:01.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 68719476739 2026-02-13T07:05:01.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-68719476739' 2026-02-13T07:05:01.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:01.902 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T07:05:01.903 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:01.903 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:05:01.904 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-13T07:05:01.904 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:01.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-13T07:05:01.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T07:05:01.905 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836484 2026-02-13T07:05:01.906 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:05:02.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T07:05:02.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T07:05:03.133 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T07:05:03.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:05:03.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-13T07:05:03.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:03.362 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-68719476739 2026-02-13T07:05:03.362 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:03.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T07:05:03.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-68719476739 2026-02-13T07:05:03.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:03.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=68719476739 2026-02-13T07:05:03.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 68719476739' 2026-02-13T07:05:03.364 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 68719476739 2026-02-13T07:05:03.365 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T07:05:03.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 68719476739 -lt 68719476739 2026-02-13T07:05:03.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: get_num_pgs 2026-02-13T07:05:03.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:03.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:03.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: test 4 == 0 2026-02-13T07:05:03.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1742: wait_for_peered: true 2026-02-13T07:05:03.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: get_num_active_or_peered 2026-02-13T07:05:03.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-13T07:05:03.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-13T07:05:03.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-13T07:05:03.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-13T07:05:03.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-13T07:05:04.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: cur_peered=4 2026-02-13T07:05:04.132 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: get_num_pgs 2026-02-13T07:05:04.132 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:04.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:04.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: test 4 = 4 2026-02-13T07:05:04.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: break 2026-02-13T07:05:04.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1762: wait_for_peered: return 0 2026-02-13T07:05:04.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1776: test_wait_for_peered: teardown td/ceph-helpers 2026-02-13T07:05:04.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:05:04.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:04.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:05:04.451 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:04.451 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:04.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:04.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:04.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:04.557 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:04.557 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:04.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:04.558 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:04.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:04.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:04.560 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:04.561 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:04.561 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:04.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:04.562 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:04.562 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:04.563 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:04.563 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:05:04.569 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:04.569 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:04.569 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:04.569 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:05:04.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:04.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:04.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:05:04.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_wait_for_scrub td/ceph-helpers 2026-02-13T07:05:04.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: test_wait_for_scrub: local dir=td/ceph-helpers 2026-02-13T07:05:04.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2079: test_wait_for_scrub: setup td/ceph-helpers 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:04.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:04.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:04.573 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:04.573 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:04.573 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:04.574 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:04.574 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:04.575 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:04.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:04.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:04.576 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:04.576 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:04.577 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:04.577 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:04.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:05:04.579 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:04.579 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:04.579 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:04.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:05:04.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:04.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:04.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:05:04.581 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:05:04.581 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:04.581 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:04.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:05:04.582 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:05:04.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:05:04.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:05:04.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:05:04.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: test_wait_for_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:05:04.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:05:04.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:05:04.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:05:04.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:05:04.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:05:04.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:05:04.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:05:04.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:04.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:04.604 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:04.604 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:04.604 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:04.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:05:04.605 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:05:04.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:05:04.632 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:05:04.632 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:05:04.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:05:04.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:05:04.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:05:04.633 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:05:04.633 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:05:04.633 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:05:04.634 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:05:04.634 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:04.634 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:04.634 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:05:04.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:05:04.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:05:04.677 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:05:04.677 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:05:04.677 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:05:04.677 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:05:04.678 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:05:04.679 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:05:04.679 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:05:04.679 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:05:04.679 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:05:04.679 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:04.679 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:04.679 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:05:04.679 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:05:04.679 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:05:04.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2081: test_wait_for_scrub: run_mgr td/ceph-helpers x 2026-02-13T07:05:04.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T07:05:04.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:05:04.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:05:04.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:05:04.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T07:05:04.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:05:04.824 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:05:04.824 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:04.824 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:04.824 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:04.824 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:04.824 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:04.824 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:05:04.825 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:05:04.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:05:04.845 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2082: test_wait_for_scrub: run_osd td/ceph-helpers 0 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:05:04.846 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:05:04.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:05:04.848 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:05:04.849 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f99a3ff4-cc4b-4812-9525-54b1bd478a34 2026-02-13T07:05:04.849 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f99a3ff4-cc4b-4812-9525-54b1bd478a34' 2026-02-13T07:05:04.849 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 f99a3ff4-cc4b-4812-9525-54b1bd478a34 2026-02-13T07:05:04.850 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:05:04.859 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAgzY5pyQ8KMxAAMXKT1LMsdPPRc628tt7J7g== 2026-02-13T07:05:04.859 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAgzY5pyQ8KMxAAMXKT1LMsdPPRc628tt7J7g=="}' 2026-02-13T07:05:04.859 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f99a3ff4-cc4b-4812-9525-54b1bd478a34 -i td/ceph-helpers/0/new.json 2026-02-13T07:05:04.965 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:04.973 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T07:05:04.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAgzY5pyQ8KMxAAMXKT1LMsdPPRc628tt7J7g== --osd-uuid f99a3ff4-cc4b-4812-9525-54b1bd478a34 2026-02-13T07:05:04.992 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:04.990+0000 7fe62e383900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:04.994 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:04.992+0000 7fe62e383900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:04.994 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:04.993+0000 7fe62e383900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:04.995 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:04.993+0000 7fe62e383900 -1 bdev(0x5654f081cc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:05:04.995 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:04.993+0000 7fe62e383900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T07:05:05.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T07:05:05.378 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:05:05.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:05:05.379 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:05:05.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:05:05.485 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:05:05.485 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:05:05.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:05:05.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:05:05.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:05:05.486 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:05:05.503 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:05.502+0000 7f3f97221900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:05.505 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:05.503+0000 7f3f97221900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:05.506 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:05.504+0000 7f3f97221900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:05.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:05:05.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:05:05.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:05:05.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:05:05.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:05:05.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:05.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:05:05.590 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:05.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:05.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:05.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:05.730 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:05.728+0000 7f3f97221900 -1 Falling back to public interface 2026-02-13T07:05:06.041 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:06.039+0000 7f3f97221900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:05:06.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:06.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:06.690 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:06.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:05:06.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:06.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:06.788 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1785819857,v1:127.0.0.1:6801/1785819857] [v2:127.0.0.1:6802/1785819857,v1:127.0.0.1:6803/1785819857] exists,up f99a3ff4-cc4b-4812-9525-54b1bd478a34 2026-02-13T07:05:06.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:05:06.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:05:06.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:05:06.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2083: test_wait_for_scrub: create_rbd_pool 2026-02-13T07:05:06.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T07:05:06.888 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T07:05:06.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T07:05:06.895 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T07:05:07.176 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T07:05:07.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T07:05:08.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T07:05:08.467 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2084: test_wait_for_scrub: wait_for_clean 2026-02-13T07:05:08.467 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:05:08.467 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:05:08.467 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:05:08.467 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T07:05:08.468 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:05:08.468 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:05:08.468 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:05:08.468 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:05:08.469 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:05:08.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:05:08.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:05:08.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:05:08.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:05:08.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:05:08.550 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:05:08.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T07:05:08.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:05:08.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:08.775 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:05:08.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T07:05:08.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T07:05:08.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T07:05:08.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:08.842 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T07:05:08.842 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:08.842 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:05:08.843 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T07:05:08.843 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:08.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T07:05:08.844 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T07:05:08.844 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T07:05:08.844 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:05:09.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T07:05:09.070 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:05:09.070 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:09.070 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:09.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T07:05:09.380 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:05:09.380 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:05:09.381 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:05:09.381 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:05:09.381 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:05:09.381 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:05:09.381 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:05:09.603 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T07:05:09.603 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:05:09.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:09.604 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:09.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T07:05:09.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T07:05:09.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T07:05:09.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2085: test_wait_for_scrub: local pgid=1.0 2026-02-13T07:05:09.932 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2086: test_wait_for_scrub: ceph pg repair 1.0 2026-02-13T07:05:10.144 INFO:tasks.workunit.client.0.trial040.stderr:instructing pg 1.0 on osd.0 to repair 2026-02-13T07:05:10.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2087: test_wait_for_scrub: get_last_scrub_stamp 1.0 2026-02-13T07:05:10.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:05:10.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:05:10.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:05:10.152 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:05:10.375 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2087: test_wait_for_scrub: local last_scrub=2026-02-13T07:05:07.113745+0000 2026-02-13T07:05:10.375 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2088: test_wait_for_scrub: wait_for_scrub 1.0 2026-02-13T07:05:07.113745+0000 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-13T07:05:07.113745+0000 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:05:10.376 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:05:10.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:05:07.113745+0000 '>' 2026-02-13T07:05:07.113745+0000 2026-02-13T07:05:10.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:05:11.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:05:11.601 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T07:05:11.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:05:11.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:05:11.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:05:11.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:05:11.602 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:05:11.824 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:05:07.113745+0000 '>' 2026-02-13T07:05:07.113745+0000 2026-02-13T07:05:11.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:05:12.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:05:12.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T07:05:12.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:05:12.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:05:12.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:05:12.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:05:12.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:05:13.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:05:10.908955+0000 '>' 2026-02-13T07:05:07.113745+0000 2026-02-13T07:05:13.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-13T07:05:13.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2089: test_wait_for_scrub: kill_daemons td/ceph-helpers KILL osd 2026-02-13T07:05:13.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:13.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:13.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:13.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:13.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:13.156 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:13.157 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2090: test_wait_for_scrub: get_last_scrub_stamp 1.0 2026-02-13T07:05:13.157 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:05:13.157 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:05:13.157 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:05:13.158 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:05:13.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2090: test_wait_for_scrub: last_scrub=2026-02-13T07:05:10.908955+0000 2026-02-13T07:05:13.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2091: test_wait_for_scrub: TIMEOUT=2 2026-02-13T07:05:13.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2091: test_wait_for_scrub: wait_for_scrub 1.0 2026-02-13T07:05:10.908955+0000 2026-02-13T07:05:13.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-13T07:05:13.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-13T07:05:10.908955+0000 2026-02-13T07:05:13.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-13T07:05:13.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-13T07:05:13.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 2 )) 2026-02-13T07:05:13.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:05:13.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:05:13.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:05:13.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:05:13.388 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:05:13.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:05:10.908955+0000 '>' 2026-02-13T07:05:10.908955+0000 2026-02-13T07:05:13.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:05:14.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:05:14.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 2 )) 2026-02-13T07:05:14.609 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T07:05:14.609 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T07:05:14.609 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T07:05:14.609 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T07:05:14.609 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T07:05:14.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T07:05:10.908955+0000 '>' 2026-02-13T07:05:10.908955+0000 2026-02-13T07:05:14.829 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T07:05:15.830 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T07:05:15.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 2 )) 2026-02-13T07:05:15.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: return 1 2026-02-13T07:05:15.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2092: test_wait_for_scrub: teardown td/ceph-helpers 2026-02-13T07:05:15.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:05:15.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:15.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:05:15.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:15.831 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:15.831 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:15.832 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:15.832 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:15.939 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:15.939 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:15.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:15.940 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:15.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:15.942 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:15.942 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:15.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:15.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:15.943 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:15.944 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:15.945 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:15.945 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:15.946 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:05:15.949 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:15.949 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:15.949 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:15.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:05:15.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:15.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: for func in $funcs 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: run_tests: test_wait_osd_id_used_by_pgs td/ceph-helpers 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1492: test_wait_osd_id_used_by_pgs: local dir=td/ceph-helpers 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1494: test_wait_osd_id_used_by_pgs: setup td/ceph-helpers 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:15.952 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:15.953 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:15.953 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:15.953 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:15.953 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:15.953 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:15.954 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:15.954 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:15.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:15.955 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:15.955 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:15.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:15.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:15.956 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:15.956 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:15.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:15.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:15.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:05:15.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:15.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:15.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:15.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:05:15.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:15.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:15.960 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T07:05:15.961 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:05:15.961 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:15.961 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:15.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.63910 2026-02-13T07:05:15.962 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:05:15.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:05:15.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:05:15.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T07:05:15.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1495: test_wait_osd_id_used_by_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:05:15.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T07:05:15.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:05:15.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:05:15.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:05:15.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T07:05:15.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:05:15.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:05:15.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:15.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:15.985 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:15.985 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:15.985 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:15.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:05:15.986 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T07:05:16.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:05:16.015 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:05:16.015 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:05:16.015 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:05:16.015 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:05:16.015 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:05:16.016 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:05:16.016 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:05:16.016 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:05:16.016 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:05:16.016 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:16.017 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:16.017 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:05:16.017 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:05:16.017 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get fsid 2026-02-13T07:05:16.060 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:05:16.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:05:16.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:05:16.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:05:16.061 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:05:16.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:05:16.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:05:16.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:05:16.062 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:05:16.063 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:16.063 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:16.063 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.63910/ceph-mon.a.asok 2026-02-13T07:05:16.063 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:05:16.063 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.63910/ceph-mon.a.asok config get mon_host 2026-02-13T07:05:16.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1496: test_wait_osd_id_used_by_pgs: run_mgr td/ceph-helpers x 2026-02-13T07:05:16.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T07:05:16.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:05:16.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:05:16.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:05:16.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T07:05:16.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:05:16.208 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:05:16.208 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:16.208 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:16.208 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:16.208 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:16.208 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:16.209 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:05:16.209 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:05:16.209 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:05:16.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1497: test_wait_osd_id_used_by_pgs: run_osd td/ceph-helpers 0 2026-02-13T07:05:16.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T07:05:16.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:05:16.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:05:16.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:05:16.230 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T07:05:16.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:05:16.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:05:16.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:05:16.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:05:16.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:05:16.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:05:16.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T07:05:16.232 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:05:16.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=337d0d83-67b7-45da-9d41-984ccf79b17d 2026-02-13T07:05:16.233 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 337d0d83-67b7-45da-9d41-984ccf79b17d 2026-02-13T07:05:16.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 337d0d83-67b7-45da-9d41-984ccf79b17d' 2026-02-13T07:05:16.233 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:05:16.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAszY5pWANHDhAAaOgr57l7qfCPuPQkrhO+wA== 2026-02-13T07:05:16.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAszY5pWANHDhAAaOgr57l7qfCPuPQkrhO+wA=="}' 2026-02-13T07:05:16.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 337d0d83-67b7-45da-9d41-984ccf79b17d -i td/ceph-helpers/0/new.json 2026-02-13T07:05:16.345 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:16.351 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T07:05:16.352 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAszY5pWANHDhAAaOgr57l7qfCPuPQkrhO+wA== --osd-uuid 337d0d83-67b7-45da-9d41-984ccf79b17d 2026-02-13T07:05:16.370 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:16.368+0000 7f5986dac900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:16.371 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:16.369+0000 7f5986dac900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:16.372 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:16.370+0000 7f5986dac900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:16.373 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:16.370+0000 7f5986dac900 -1 bdev(0x560d2ad16c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:05:16.373 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:16.371+0000 7f5986dac900 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T07:05:16.797 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T07:05:16.797 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:05:16.797 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:05:16.798 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:05:16.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:05:16.903 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:05:16.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:05:16.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1f27ccbb-a9b1-48b5-8639-a657d3ca6ca2 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.63910/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:05:16.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:05:16.904 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:05:16.905 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:05:16.920 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:16.918+0000 7ff099214900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:16.922 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:16.920+0000 7ff099214900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:16.923 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:16.921+0000 7ff099214900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:17.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:05:17.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:05:17.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:05:17.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:05:17.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:05:17.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:17.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:05:17.005 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:17.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:17.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:17.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:17.122 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:17.121+0000 7ff099214900 -1 Falling back to public interface 2026-02-13T07:05:17.478 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:17.475+0000 7ff099214900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:05:18.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:18.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:18.107 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:18.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:05:18.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:18.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:18.209 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1396740741,v1:127.0.0.1:6801/1396740741] [v2:127.0.0.1:6802/1396740741,v1:127.0.0.1:6803/1396740741] exists,up 337d0d83-67b7-45da-9d41-984ccf79b17d 2026-02-13T07:05:18.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:05:18.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:05:18.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:05:18.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1498: test_wait_osd_id_used_by_pgs: create_rbd_pool 2026-02-13T07:05:18.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T07:05:18.306 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' does not exist 2026-02-13T07:05:18.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T07:05:18.313 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T07:05:18.559 INFO:tasks.workunit.client.0.trial040.stderr:pool 'rbd' already exists 2026-02-13T07:05:18.566 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T07:05:19.568 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T07:05:19.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1499: test_wait_osd_id_used_by_pgs: wait_for_clean 2026-02-13T07:05:19.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:05:19.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:05:19.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:05:19.852 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T07:05:19.852 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:05:19.852 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:05:19.852 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:05:19.852 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:05:19.852 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:05:19.934 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:05:19.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:05:19.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:05:19.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:05:19.935 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:05:19.935 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:05:20.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T07:05:20.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:05:20.159 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:20.159 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:05:20.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T07:05:20.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T07:05:20.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T07:05:20.224 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:20.224 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T07:05:20.224 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:20.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:05:20.225 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-13T07:05:20.226 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:20.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-13T07:05:20.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T07:05:20.227 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836483 2026-02-13T07:05:20.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:05:20.463 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T07:05:20.463 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:05:20.463 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:20.464 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:20.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T07:05:20.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:05:20.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:05:20.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:05:20.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:05:20.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:05:20.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:05:20.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:05:21.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T07:05:21.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:05:21.006 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:21.006 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:21.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T07:05:21.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T07:05:21.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T07:05:21.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1500: test_wait_osd_id_used_by_pgs: wait_osd_id_used_by_pgs 0 8 2026-02-13T07:05:21.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1475: wait_osd_id_used_by_pgs: local id=0 2026-02-13T07:05:21.326 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1476: wait_osd_id_used_by_pgs: local count=8 2026-02-13T07:05:21.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1478: wait_osd_id_used_by_pgs: status=1 2026-02-13T07:05:21.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1479: wait_osd_id_used_by_pgs: (( i=0 )) 2026-02-13T07:05:21.327 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1479: wait_osd_id_used_by_pgs: (( i < 300 / 5 )) 2026-02-13T07:05:21.327 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:21.328 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1480: wait_osd_id_used_by_pgs: echo 0 2026-02-13T07:05:21.328 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1481: wait_osd_id_used_by_pgs: get_osd_id_used_by_pgs 2026-02-13T07:05:21.328 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1481: wait_osd_id_used_by_pgs: grep -c 0 2026-02-13T07:05:21.328 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: ceph --format json pg dump pgs 2026-02-13T07:05:21.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: jq '.pg_stats | .[] | .up[], .acting[]' 2026-02-13T07:05:21.329 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: sort 2026-02-13T07:05:21.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1481: wait_osd_id_used_by_pgs: test 8 = 8 2026-02-13T07:05:21.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1484: wait_osd_id_used_by_pgs: status=0 2026-02-13T07:05:21.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1485: wait_osd_id_used_by_pgs: break 2026-02-13T07:05:21.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1488: wait_osd_id_used_by_pgs: return 0 2026-02-13T07:05:21.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1501: test_wait_osd_id_used_by_pgs: TIMEOUT=1 2026-02-13T07:05:21.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1501: test_wait_osd_id_used_by_pgs: wait_osd_id_used_by_pgs 123 5 2026-02-13T07:05:21.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1475: wait_osd_id_used_by_pgs: local id=123 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1476: wait_osd_id_used_by_pgs: local count=5 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1478: wait_osd_id_used_by_pgs: status=1 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1479: wait_osd_id_used_by_pgs: (( i=0 )) 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1479: wait_osd_id_used_by_pgs: (( i < 1 / 5 )) 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1488: wait_osd_id_used_by_pgs: return 1 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1502: test_wait_osd_id_used_by_pgs: teardown td/ceph-helpers 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:21.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:21.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:21.552 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:21.656 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:21.656 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:21.657 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:21.657 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:21.658 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:21.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:21.659 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:21.660 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:21.660 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:21.660 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:21.661 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:21.661 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:21.662 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T07:05:21.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:21.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:21.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.63910 2026-02-13T07:05:21.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.63910 2026-02-13T07:05:21.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:21.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:21.667 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2437: : exit 0 2026-02-13T07:05:21.668 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T07:05:21.668 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T07:05:21.694 INFO:tasks.workunit:Running workunit misc/test-snaptrim-stats.sh... 2026-02-13T07:05:21.695 DEBUG:teuthology.orchestra.run.trial040:workunit test misc/test-snaptrim-stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh 2026-02-13T07:05:21.759 INFO:tasks.workunit.client.0.trial040.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/test-snaptrim-stats 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:21: run: local dir=td/test-snaptrim-stats 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:22: run: shift 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:24: run: export CEPH_MON=127.0.0.1:7124 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:24: run: CEPH_MON=127.0.0.1:7124 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:25: run: export CEPH_ARGS 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:26: run: uuidgen 2026-02-13T07:05:21.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:26: run: CEPH_ARGS+='--fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none ' 2026-02-13T07:05:21.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-13T07:05:21.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:28: run: CEPH_ARGS+='--debug-bluestore 20 ' 2026-02-13T07:05:21.764 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:30: run: set 2026-02-13T07:05:21.764 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:30: run: local 'funcs=TEST_snaptrim_stats 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:TEST_snaptrim_stats_multiple_snaps' 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:31: run: for func in $funcs 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:32: run: setup td/test-snaptrim-stats 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-snaptrim-stats 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-snaptrim-stats 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-snaptrim-stats 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-snaptrim-stats KILL 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:21.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:21.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:21.768 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:21.769 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:21.769 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:21.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:21.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:21.770 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:21.771 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:21.771 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:21.771 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:21.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:21.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:21.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:21.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-snaptrim-stats 2026-02-13T07:05:21.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:21.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:21.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:21.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.150893 2026-02-13T07:05:21.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:21.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:21.775 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-snaptrim-stats 2026-02-13T07:05:21.776 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:05:21.776 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:21.776 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:21.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.150893 2026-02-13T07:05:21.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:05:21.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-snaptrim-stats 1' TERM HUP INT 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:33: run: TEST_snaptrim_stats td/test-snaptrim-stats 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:39: TEST_snaptrim_stats: local dir=td/test-snaptrim-stats 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:40: TEST_snaptrim_stats: local poolname=test 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:41: TEST_snaptrim_stats: local OSDS=3 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:42: TEST_snaptrim_stats: local PGNUM=8 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:43: TEST_snaptrim_stats: local PGPNUM=8 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:44: TEST_snaptrim_stats: local objects=10 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:45: TEST_snaptrim_stats: local WAIT_FOR_UPDATE=10 2026-02-13T07:05:21.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:47: TEST_snaptrim_stats: setup td/test-snaptrim-stats 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-snaptrim-stats 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-snaptrim-stats 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-snaptrim-stats 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-snaptrim-stats KILL 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:21.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:21.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:21.782 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:21.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:21.783 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:21.783 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:21.783 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:21.783 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:21.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:21.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:21.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:21.784 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:21.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:21.785 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:21.785 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-snaptrim-stats 2026-02-13T07:05:21.786 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:21.786 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:21.786 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:21.786 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.150893 2026-02-13T07:05:21.787 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:21.787 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:21.787 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-snaptrim-stats 2026-02-13T07:05:21.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:05:21.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:21.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:21.788 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.150893 2026-02-13T07:05:21.789 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:05:21.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:05:21.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:05:21.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-snaptrim-stats 1' TERM HUP INT 2026-02-13T07:05:21.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:48: TEST_snaptrim_stats: run_mon td/test-snaptrim-stats a --osd_pool_default_size=3 2026-02-13T07:05:21.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-snaptrim-stats 2026-02-13T07:05:21.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:05:21.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:05:21.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:05:21.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-snaptrim-stats/a 2026-02-13T07:05:21.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-snaptrim-stats/a --run-dir=td/test-snaptrim-stats --osd_pool_default_size=3 2026-02-13T07:05:21.810 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:05:21.810 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:21.810 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:21.810 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:21.811 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:21.811 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:21.811 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:21.811 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-snaptrim-stats/a '--log-file=td/test-snaptrim-stats/$name.log' '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --mon-cluster-log-file=td/test-snaptrim-stats/log --run-dir=td/test-snaptrim-stats '--pid-file=td/test-snaptrim-stats/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-13T07:05:21.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:05:21.838 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:05:21.838 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:05:21.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:05:21.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:05:21.839 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:05:21.839 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:05:21.839 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:05:21.839 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:05:21.840 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:05:21.840 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:21.840 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:21.840 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.150893/ceph-mon.a.asok 2026-02-13T07:05:21.840 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:05:21.840 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.150893/ceph-mon.a.asok config get fsid 2026-02-13T07:05:21.885 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:05:21.885 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:05:21.885 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:05:21.885 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:05:21.885 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:05:21.886 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:05:21.886 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:05:21.887 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:05:21.887 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:05:21.887 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:21.887 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:21.887 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.150893/ceph-mon.a.asok 2026-02-13T07:05:21.887 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:05:21.887 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.150893/ceph-mon.a.asok config get mon_host 2026-02-13T07:05:21.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:49: TEST_snaptrim_stats: run_mgr td/test-snaptrim-stats x 2026-02-13T07:05:21.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-snaptrim-stats 2026-02-13T07:05:21.930 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:05:21.930 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:05:21.930 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:05:21.930 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-snaptrim-stats/x 2026-02-13T07:05:21.930 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:05:22.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:05:22.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:22.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:22.027 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:22.027 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:22.027 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:22.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:22.028 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:05:22.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-snaptrim-stats/x '--log-file=td/test-snaptrim-stats/$name.log' '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --run-dir=td/test-snaptrim-stats '--pid-file=td/test-snaptrim-stats/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:05:22.045 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:50: TEST_snaptrim_stats: expr 3 - 1 2026-02-13T07:05:22.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:50: TEST_snaptrim_stats: seq 0 2 2026-02-13T07:05:22.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:50: TEST_snaptrim_stats: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-13T07:05:22.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:52: TEST_snaptrim_stats: run_osd td/test-snaptrim-stats 0 --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:22.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-snaptrim-stats 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-snaptrim-stats/0 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 ' 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-snaptrim-stats/0' 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-snaptrim-stats/0/journal' 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:05:22.050 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-snaptrim-stats' 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:05:22.051 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-snaptrim-stats/$name.log' 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-snaptrim-stats/$name.pid' 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-snaptrim-stats/0 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f7ad3992-eb25-4f32-86e5-27a455c26ebb 2026-02-13T07:05:22.052 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f7ad3992-eb25-4f32-86e5-27a455c26ebb' 2026-02-13T07:05:22.053 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 f7ad3992-eb25-4f32-86e5-27a455c26ebb 2026-02-13T07:05:22.053 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:05:22.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAyzY5pmzZ1AxAAt0ZCZu4tRY5Fz1Gn3TiMRA== 2026-02-13T07:05:22.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAyzY5pmzZ1AxAAt0ZCZu4tRY5Fz1Gn3TiMRA=="}' 2026-02-13T07:05:22.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f7ad3992-eb25-4f32-86e5-27a455c26ebb -i td/test-snaptrim-stats/0/new.json 2026-02-13T07:05:22.163 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:22.169 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-snaptrim-stats/0/new.json 2026-02-13T07:05:22.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/0 --osd-journal=td/test-snaptrim-stats/0/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --mkfs --key AQAyzY5pmzZ1AxAAt0ZCZu4tRY5Fz1Gn3TiMRA== --osd-uuid f7ad3992-eb25-4f32-86e5-27a455c26ebb 2026-02-13T07:05:22.188 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:22.186+0000 7ff89bc12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:22.190 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:22.188+0000 7ff89bc12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:22.191 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:22.189+0000 7ff89bc12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:22.191 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:22.189+0000 7ff89bc12900 -1 bdev(0x55b38ca8ec00 td/test-snaptrim-stats/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:05:22.191 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:22.189+0000 7ff89bc12900 -1 bluestore(td/test-snaptrim-stats/0) _read_fsid unparsable uuid 2026-02-13T07:05:22.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-snaptrim-stats/0/keyring 2026-02-13T07:05:22.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:05:22.592 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:05:22.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:05:22.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-snaptrim-stats/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:05:22.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:05:22.696 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:05:22.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/0 --osd-journal=td/test-snaptrim-stats/0/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:22.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:05:22.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:05:22.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:05:22.712 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:22.710+0000 7f36eb465900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:22.713 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:22.712+0000 7f36eb465900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:22.714 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:22.712+0000 7f36eb465900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:22.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:05:22.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:05:22.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:05:22.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:05:22.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:05:22.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:22.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:05:22.794 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:22.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:22.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:22.891 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:22.939 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:22.937+0000 7f36eb465900 -1 Falling back to public interface 2026-02-13T07:05:23.295 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:23.293+0000 7f36eb465900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:05:23.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:23.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:23.893 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:23.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:05:23.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:23.894 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:23.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:24.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:24.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:24.996 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:05:24.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-13T07:05:24.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:24.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:25.220 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3321505917,v1:127.0.0.1:6801/3321505917] [v2:127.0.0.1:6802/3321505917,v1:127.0.0.1:6803/3321505917] exists,up f7ad3992-eb25-4f32-86e5-27a455c26ebb 2026-02-13T07:05:25.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:05:25.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:05:25.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:05:25.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:50: TEST_snaptrim_stats: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-13T07:05:25.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:52: TEST_snaptrim_stats: run_osd td/test-snaptrim-stats 1 --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:25.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-snaptrim-stats 2026-02-13T07:05:25.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:05:25.225 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-snaptrim-stats/1 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 ' 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-snaptrim-stats/1' 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-snaptrim-stats/1/journal' 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-snaptrim-stats' 2026-02-13T07:05:25.226 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-snaptrim-stats/$name.log' 2026-02-13T07:05:25.227 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-snaptrim-stats/$name.pid' 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-snaptrim-stats/1 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9a3b5342-9f8d-4324-8fba-e7e109fa6bd4 2026-02-13T07:05:25.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 9a3b5342-9f8d-4324-8fba-e7e109fa6bd4' 2026-02-13T07:05:25.229 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 9a3b5342-9f8d-4324-8fba-e7e109fa6bd4 2026-02-13T07:05:25.229 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:05:25.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA1zY5pRNDpDRAAdYqUCnjvMt4RdA8r70iCIw== 2026-02-13T07:05:25.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA1zY5pRNDpDRAAdYqUCnjvMt4RdA8r70iCIw=="}' 2026-02-13T07:05:25.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9a3b5342-9f8d-4324-8fba-e7e109fa6bd4 -i td/test-snaptrim-stats/1/new.json 2026-02-13T07:05:25.469 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:25.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-snaptrim-stats/1/new.json 2026-02-13T07:05:25.478 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/1 --osd-journal=td/test-snaptrim-stats/1/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --mkfs --key AQA1zY5pRNDpDRAAdYqUCnjvMt4RdA8r70iCIw== --osd-uuid 9a3b5342-9f8d-4324-8fba-e7e109fa6bd4 2026-02-13T07:05:25.495 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:25.493+0000 7f3fcfd79900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:25.496 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:25.495+0000 7f3fcfd79900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:25.497 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:25.496+0000 7f3fcfd79900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:25.498 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:25.496+0000 7f3fcfd79900 -1 bdev(0x556bd7486c00 td/test-snaptrim-stats/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:05:25.498 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:25.496+0000 7f3fcfd79900 -1 bluestore(td/test-snaptrim-stats/1) _read_fsid unparsable uuid 2026-02-13T07:05:25.833 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-snaptrim-stats/1/keyring 2026-02-13T07:05:25.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:05:25.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:05:25.835 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:05:25.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-snaptrim-stats/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:05:26.148 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:05:26.149 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:05:26.149 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/1 --osd-journal=td/test-snaptrim-stats/1/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:26.149 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:05:26.149 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:05:26.149 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:05:26.165 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:26.163+0000 7fcfdd80f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:26.166 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:26.164+0000 7fcfdd80f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:26.167 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:26.165+0000 7fcfdd80f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:26.310 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:26.308+0000 7fcfdd80f900 -1 Falling back to public interface 2026-02-13T07:05:26.372 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:05:26.372 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:05:26.372 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:05:26.372 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:05:26.373 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:05:26.373 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:26.373 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:05:26.373 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:26.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:26.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:05:26.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:26.632 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:26.630+0000 7fcfdd80f900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:05:27.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:27.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:27.598 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:27.598 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:05:27.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:27.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:05:27.821 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/306824445,v1:127.0.0.1:6811/306824445] [v2:127.0.0.1:6812/306824445,v1:127.0.0.1:6813/306824445] exists,up 9a3b5342-9f8d-4324-8fba-e7e109fa6bd4 2026-02-13T07:05:27.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:05:27.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:05:27.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:05:27.825 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:50: TEST_snaptrim_stats: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:52: TEST_snaptrim_stats: run_osd td/test-snaptrim-stats 2 --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-snaptrim-stats 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-snaptrim-stats/2 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 ' 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-snaptrim-stats/2' 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-snaptrim-stats/2/journal' 2026-02-13T07:05:27.826 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-snaptrim-stats' 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:05:27.827 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-snaptrim-stats/$name.log' 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-snaptrim-stats/$name.pid' 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-snaptrim-stats/2 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:05:27.828 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1b38f07a-52b0-436f-8597-d7c9b24511ec 2026-02-13T07:05:27.829 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 1b38f07a-52b0-436f-8597-d7c9b24511ec 2026-02-13T07:05:27.829 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 1b38f07a-52b0-436f-8597-d7c9b24511ec' 2026-02-13T07:05:27.829 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:05:27.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA3zY5pDcHDMRAAb/wBEgiHflhsC7qkYauvwg== 2026-02-13T07:05:27.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA3zY5pDcHDMRAAb/wBEgiHflhsC7qkYauvwg=="}' 2026-02-13T07:05:27.837 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1b38f07a-52b0-436f-8597-d7c9b24511ec -i td/test-snaptrim-stats/2/new.json 2026-02-13T07:05:28.065 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:05:28.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-snaptrim-stats/2/new.json 2026-02-13T07:05:28.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/2 --osd-journal=td/test-snaptrim-stats/2/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --mkfs --key AQA3zY5pDcHDMRAAb/wBEgiHflhsC7qkYauvwg== --osd-uuid 1b38f07a-52b0-436f-8597-d7c9b24511ec 2026-02-13T07:05:28.091 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:28.089+0000 7f04a4173900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:28.092 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:28.090+0000 7f04a4173900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:28.093 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:28.091+0000 7f04a4173900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:28.093 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:28.091+0000 7f04a4173900 -1 bdev(0x55c4fae24c00 td/test-snaptrim-stats/2/block) open stat got: (1) Operation not permitted 2026-02-13T07:05:28.094 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:28.091+0000 7f04a4173900 -1 bluestore(td/test-snaptrim-stats/2) _read_fsid unparsable uuid 2026-02-13T07:05:28.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-snaptrim-stats/2/keyring 2026-02-13T07:05:28.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:05:28.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T07:05:28.412 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T07:05:28.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-snaptrim-stats/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:05:28.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T07:05:28.731 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:05:28.731 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/2 --osd-journal=td/test-snaptrim-stats/2/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:28.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:05:28.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:05:28.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:05:28.747 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:28.745+0000 7f71f8f42900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:28.749 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:28.747+0000 7f71f8f42900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:28.750 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:28.748+0000 7f71f8f42900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:28.888 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:28.886+0000 7f71f8f42900 -1 Falling back to public interface 2026-02-13T07:05:28.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T07:05:28.957 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:05:28.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:05:28.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:05:28.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:05:28.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:28.958 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:05:28.958 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:28.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:28.959 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:05:29.170 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:29.168+0000 7f71f8f42900 -1 osd.2 0 log_to_monitors true 2026-02-13T07:05:29.191 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:30.192 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:30.193 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:30.193 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:30.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:05:30.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:30.194 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:05:30.416 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1242612216,v1:127.0.0.1:6819/1242612216] [v2:127.0.0.1:6820/1242612216,v1:127.0.0.1:6821/1242612216] exists,up 1b38f07a-52b0-436f-8597-d7c9b24511ec 2026-02-13T07:05:30.417 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:05:30.417 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:05:30.417 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:05:30.417 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:56: TEST_snaptrim_stats: ceph osd set noscrub 2026-02-13T07:05:30.701 INFO:tasks.workunit.client.0.trial040.stderr:noscrub is set 2026-02-13T07:05:30.711 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:57: TEST_snaptrim_stats: ceph osd set nodeep-scrub 2026-02-13T07:05:31.003 INFO:tasks.workunit.client.0.trial040.stderr:nodeep-scrub is set 2026-02-13T07:05:31.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:60: TEST_snaptrim_stats: create_pool test 8 8 2026-02-13T07:05:31.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 8 8 2026-02-13T07:05:31.269 INFO:tasks.workunit.client.0.trial040.stderr:pool 'test' already exists 2026-02-13T07:05:31.278 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T07:05:32.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:61: TEST_snaptrim_stats: wait_for_clean 2026-02-13T07:05:32.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:05:32.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:05:32.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:05:32.281 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T07:05:32.281 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:05:32.282 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:05:32.282 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:05:32.282 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:05:32.282 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:05:32.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:05:32.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:05:32.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:05:32.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:05:32.366 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:05:32.366 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:05:32.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T07:05:32.592 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T07:05:32.593 INFO:tasks.workunit.client.0.trial040.stderr:2' 2026-02-13T07:05:32.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:05:32.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:32.593 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:05:32.654 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T07:05:32.654 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T07:05:32.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T07:05:32.655 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:32.655 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T07:05:32.718 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-13T07:05:32.719 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-13T07:05:32.719 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672963' 2026-02-13T07:05:32.719 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:32.719 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T07:05:32.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-13T07:05:32.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-13T07:05:32.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672963 2-64424509443' 2026-02-13T07:05:32.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:32.782 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T07:05:32.783 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:32.783 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:05:32.784 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-13T07:05:32.784 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:32.785 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-13T07:05:32.785 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836484 2026-02-13T07:05:32.786 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T07:05:32.786 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:05:33.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T07:05:33.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T07:05:34.019 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T07:05:34.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:05:34.247 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-13T07:05:34.247 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:34.248 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-13T07:05:34.248 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:34.248 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T07:05:34.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672963 2026-02-13T07:05:34.249 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:34.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672963 2026-02-13T07:05:34.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-13T07:05:34.251 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672963 2026-02-13T07:05:34.251 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T07:05:34.487 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672963 2026-02-13T07:05:34.487 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:34.488 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509443 2026-02-13T07:05:34.488 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:34.488 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T07:05:34.489 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509443 2026-02-13T07:05:34.489 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:34.490 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509443 2026-02-13T07:05:34.490 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 64424509443 2026-02-13T07:05:34.491 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-13T07:05:34.491 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T07:05:34.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-13T07:05:34.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:05:34.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:34.716 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:35.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 8 == 0 2026-02-13T07:05:35.029 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:05:35.030 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:05:35.031 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:05:35.031 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:05:35.031 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:05:35.031 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:05:35.031 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:05:35.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=8 2026-02-13T07:05:35.251 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:05:35.251 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:35.251 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:35.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 = 8 2026-02-13T07:05:35.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T07:05:35.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T07:05:35.588 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:62: TEST_snaptrim_stats: ceph osd dump 2026-02-13T07:05:35.588 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:62: TEST_snaptrim_stats: grep '^pool.*['\'']test['\'']' 2026-02-13T07:05:35.588 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:62: TEST_snaptrim_stats: awk '{ print $2 }' 2026-02-13T07:05:35.818 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:62: TEST_snaptrim_stats: poolid=1 2026-02-13T07:05:35.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:65: TEST_snaptrim_stats: TESTDATA=testdata.1 2026-02-13T07:05:35.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:66: TEST_snaptrim_stats: dd if=/dev/urandom of=testdata.1 bs=4096 count=1 2026-02-13T07:05:35.819 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records in 2026-02-13T07:05:35.819 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records out 2026-02-13T07:05:35.820 INFO:tasks.workunit.client.0.trial040.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 4.5504e-05 s, 90.0 MB/s 2026-02-13T07:05:35.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: seq 1 10 2026-02-13T07:05:35.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj1 testdata.1 2026-02-13T07:05:35.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.839 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj2 testdata.1 2026-02-13T07:05:35.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.857 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj3 testdata.1 2026-02-13T07:05:35.874 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.875 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj4 testdata.1 2026-02-13T07:05:35.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.893 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj5 testdata.1 2026-02-13T07:05:35.911 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.912 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj6 testdata.1 2026-02-13T07:05:35.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.929 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj7 testdata.1 2026-02-13T07:05:35.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.947 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj8 testdata.1 2026-02-13T07:05:35.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj9 testdata.1 2026-02-13T07:05:35.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:67: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:35.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:69: TEST_snaptrim_stats: rados -p test put obj10 testdata.1 2026-02-13T07:05:35.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:71: TEST_snaptrim_stats: rm -f testdata.1 2026-02-13T07:05:35.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:74: TEST_snaptrim_stats: SNAP=1 2026-02-13T07:05:35.999 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:75: TEST_snaptrim_stats: rados -p test mksnap snap1 2026-02-13T07:05:36.067 INFO:tasks.workunit.client.0.trial040.stdout:created pool test snap snap1 2026-02-13T07:05:36.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:76: TEST_snaptrim_stats: TESTDATA=testdata.2 2026-02-13T07:05:36.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:77: TEST_snaptrim_stats: dd if=/dev/urandom of=testdata.2 bs=4096 count=1 2026-02-13T07:05:36.069 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records in 2026-02-13T07:05:36.069 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records out 2026-02-13T07:05:36.070 INFO:tasks.workunit.client.0.trial040.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 4.322e-05 s, 94.8 MB/s 2026-02-13T07:05:36.070 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: seq 1 10 2026-02-13T07:05:36.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj1 testdata.2 2026-02-13T07:05:36.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj2 testdata.2 2026-02-13T07:05:36.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj3 testdata.2 2026-02-13T07:05:36.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.124 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj4 testdata.2 2026-02-13T07:05:36.142 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.143 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj5 testdata.2 2026-02-13T07:05:36.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.160 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj6 testdata.2 2026-02-13T07:05:36.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj7 testdata.2 2026-02-13T07:05:36.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.195 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj8 testdata.2 2026-02-13T07:05:36.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.213 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj9 testdata.2 2026-02-13T07:05:36.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:78: TEST_snaptrim_stats: for i in `seq 1 $objects` 2026-02-13T07:05:36.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:80: TEST_snaptrim_stats: rados -p test put obj10 testdata.2 2026-02-13T07:05:36.247 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:82: TEST_snaptrim_stats: rm -f testdata.2 2026-02-13T07:05:36.247 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:85: TEST_snaptrim_stats: rados -p test rmsnap snap1 2026-02-13T07:05:36.268 INFO:tasks.workunit.client.0.trial040.stdout:removed pool test snap snap1 2026-02-13T07:05:36.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:88: TEST_snaptrim_stats: wait_for_clean 2026-02-13T07:05:36.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:05:36.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:05:36.271 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:05:36.271 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T07:05:36.272 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:05:36.272 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:05:36.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:05:36.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:05:36.272 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:05:36.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:05:36.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:05:36.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:05:36.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:05:36.353 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:05:36.353 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:05:36.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T07:05:36.580 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T07:05:36.580 INFO:tasks.workunit.client.0.trial040.stderr:2' 2026-02-13T07:05:36.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:05:36.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:36.580 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:05:36.641 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-13T07:05:36.641 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-13T07:05:36.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-13T07:05:36.642 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:36.642 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T07:05:36.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672966 2026-02-13T07:05:36.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672966 2026-02-13T07:05:36.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672966' 2026-02-13T07:05:36.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:05:36.705 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T07:05:36.769 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509446 2026-02-13T07:05:36.769 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509446 2026-02-13T07:05:36.769 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672966 2-64424509446' 2026-02-13T07:05:36.769 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:36.770 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-13T07:05:36.770 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:36.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:05:36.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-13T07:05:36.772 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:36.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-13T07:05:36.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-13T07:05:36.773 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836487 2026-02-13T07:05:36.773 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:05:37.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-13T07:05:37.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T07:05:38.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T07:05:38.003 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:05:38.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-13T07:05:38.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:38.229 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672966 2026-02-13T07:05:38.229 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:38.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T07:05:38.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672966 2026-02-13T07:05:38.231 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:38.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672966 2026-02-13T07:05:38.232 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672966 2026-02-13T07:05:38.232 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-13T07:05:38.233 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T07:05:38.455 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672967 -lt 42949672966 2026-02-13T07:05:38.455 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:05:38.456 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509446 2026-02-13T07:05:38.456 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:05:38.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T07:05:38.457 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509446 2026-02-13T07:05:38.458 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:05:38.458 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509446 2026-02-13T07:05:38.459 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 64424509446 2026-02-13T07:05:38.459 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-13T07:05:38.460 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T07:05:38.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-13T07:05:38.684 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:05:38.685 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:38.685 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:38.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 8 == 0 2026-02-13T07:05:38.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:05:38.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:05:38.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:05:38.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:05:38.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:05:38.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:05:38.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:05:39.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=8 2026-02-13T07:05:39.211 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:05:39.211 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:05:39.211 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:05:39.521 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 = 8 2026-02-13T07:05:39.521 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T07:05:39.521 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T07:05:39.521 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:89: TEST_snaptrim_stats: sleep 10 2026-02-13T07:05:49.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:90: TEST_snaptrim_stats: local objects_trimmed=0 2026-02-13T07:05:49.522 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:91: TEST_snaptrim_stats: local snaptrim_duration_total=0.0 2026-02-13T07:05:49.523 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: expr 8 - 1 2026-02-13T07:05:49.524 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: seq 0 7 2026-02-13T07:05:49.525 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:05:49.525 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:94: TEST_snaptrim_stats: local pgid=1.0 2026-02-13T07:05:49.527 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: ceph pg 1.0 query 2026-02-13T07:05:49.527 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: jq .info.stats.objects_trimmed 2026-02-13T07:05:49.593 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: expr 0 + 2 2026-02-13T07:05:49.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: objects_trimmed=2 2026-02-13T07:05:49.595 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: bc 2026-02-13T07:05:49.596 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: ceph pg 1.0 query 2026-02-13T07:05:49.596 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: jq .info.stats.snaptrim_duration 2026-02-13T07:05:49.662 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: echo 0.0 + 0.00166607 2026-02-13T07:05:49.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: snaptrim_duration_total=.00166607 2026-02-13T07:05:49.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:05:49.663 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:94: TEST_snaptrim_stats: local pgid=1.1 2026-02-13T07:05:49.664 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: ceph pg 1.1 query 2026-02-13T07:05:49.664 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: jq .info.stats.objects_trimmed 2026-02-13T07:05:49.730 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: expr 2 + 2 2026-02-13T07:05:49.731 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: objects_trimmed=4 2026-02-13T07:05:49.732 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: bc 2026-02-13T07:05:49.732 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: ceph pg 1.1 query 2026-02-13T07:05:49.732 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: jq .info.stats.snaptrim_duration 2026-02-13T07:05:49.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: echo .00166607 + 0.00158545 2026-02-13T07:05:49.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: snaptrim_duration_total=.00325152 2026-02-13T07:05:49.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:05:49.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:94: TEST_snaptrim_stats: local pgid=1.2 2026-02-13T07:05:49.799 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: ceph pg 1.2 query 2026-02-13T07:05:49.799 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: jq .info.stats.objects_trimmed 2026-02-13T07:05:49.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: expr 4 + 0 2026-02-13T07:05:49.868 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: objects_trimmed=4 2026-02-13T07:05:49.869 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: bc 2026-02-13T07:05:49.869 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: ceph pg 1.2 query 2026-02-13T07:05:49.869 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: jq .info.stats.snaptrim_duration 2026-02-13T07:05:49.936 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: echo .00325152 + 0.00022864599999999999 2026-02-13T07:05:49.937 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: snaptrim_duration_total=.00348016599999999999 2026-02-13T07:05:49.937 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:05:49.937 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:94: TEST_snaptrim_stats: local pgid=1.3 2026-02-13T07:05:49.938 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: ceph pg 1.3 query 2026-02-13T07:05:49.938 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: jq .info.stats.objects_trimmed 2026-02-13T07:05:50.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: expr 4 + 2 2026-02-13T07:05:50.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: objects_trimmed=6 2026-02-13T07:05:50.006 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: bc 2026-02-13T07:05:50.006 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: ceph pg 1.3 query 2026-02-13T07:05:50.006 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: jq .info.stats.snaptrim_duration 2026-02-13T07:05:50.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: echo .00348016599999999999 + 0.0029744440000000001 2026-02-13T07:05:50.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: snaptrim_duration_total=.00645461000000000009 2026-02-13T07:05:50.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:05:50.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:94: TEST_snaptrim_stats: local pgid=1.4 2026-02-13T07:05:50.074 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: ceph pg 1.4 query 2026-02-13T07:05:50.075 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: jq .info.stats.objects_trimmed 2026-02-13T07:05:50.140 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: expr 6 + 1 2026-02-13T07:05:50.141 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: objects_trimmed=7 2026-02-13T07:05:50.142 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: bc 2026-02-13T07:05:50.143 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: ceph pg 1.4 query 2026-02-13T07:05:50.143 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: jq .info.stats.snaptrim_duration 2026-02-13T07:05:50.209 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: echo .00645461000000000009 + 0.0015317099999999999 2026-02-13T07:05:50.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: snaptrim_duration_total=.00798631999999999999 2026-02-13T07:05:50.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:05:50.210 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:94: TEST_snaptrim_stats: local pgid=1.5 2026-02-13T07:05:50.211 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: ceph pg 1.5 query 2026-02-13T07:05:50.211 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: jq .info.stats.objects_trimmed 2026-02-13T07:05:50.278 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: expr 7 + 1 2026-02-13T07:05:50.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: objects_trimmed=8 2026-02-13T07:05:50.280 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: bc 2026-02-13T07:05:50.281 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: ceph pg 1.5 query 2026-02-13T07:05:50.281 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: jq .info.stats.snaptrim_duration 2026-02-13T07:05:50.345 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: echo .00798631999999999999 + 0.0011142529999999999 2026-02-13T07:05:50.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: snaptrim_duration_total=.00910057299999999989 2026-02-13T07:05:50.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:05:50.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:94: TEST_snaptrim_stats: local pgid=1.6 2026-02-13T07:05:50.347 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: ceph pg 1.6 query 2026-02-13T07:05:50.347 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: jq .info.stats.objects_trimmed 2026-02-13T07:05:50.412 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: expr 8 + 0 2026-02-13T07:05:50.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: objects_trimmed=8 2026-02-13T07:05:50.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: bc 2026-02-13T07:05:50.414 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: ceph pg 1.6 query 2026-02-13T07:05:50.414 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: jq .info.stats.snaptrim_duration 2026-02-13T07:05:50.481 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: echo .00910057299999999989 + 0.0018426180000000001 2026-02-13T07:05:50.482 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: snaptrim_duration_total=.01094319099999999999 2026-02-13T07:05:50.482 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:92: TEST_snaptrim_stats: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:05:50.482 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:94: TEST_snaptrim_stats: local pgid=1.7 2026-02-13T07:05:50.483 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: ceph pg 1.7 query 2026-02-13T07:05:50.483 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: jq .info.stats.objects_trimmed 2026-02-13T07:05:50.550 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: expr 8 + 2 2026-02-13T07:05:50.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:96: TEST_snaptrim_stats: objects_trimmed=10 2026-02-13T07:05:50.552 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: bc 2026-02-13T07:05:50.552 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: ceph pg 1.7 query 2026-02-13T07:05:50.552 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: jq .info.stats.snaptrim_duration 2026-02-13T07:05:50.620 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: echo .01094319099999999999 + 0.003225562 2026-02-13T07:05:50.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:98: TEST_snaptrim_stats: snaptrim_duration_total=.01416875299999999999 2026-02-13T07:05:50.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:100: TEST_snaptrim_stats: test 10 -eq 10 2026-02-13T07:05:50.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:101: TEST_snaptrim_stats: echo '.01416875299999999999 > 0.0' 2026-02-13T07:05:50.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:101: TEST_snaptrim_stats: bc 2026-02-13T07:05:50.621 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:50.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:103: TEST_snaptrim_stats: teardown td/test-snaptrim-stats 2026-02-13T07:05:50.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-snaptrim-stats 2026-02-13T07:05:50.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:50.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-snaptrim-stats KILL 2026-02-13T07:05:50.623 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:50.623 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:50.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:50.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:50.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:50.729 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:50.729 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:50.730 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:50.730 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:50.731 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:50.731 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:50.731 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:50.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:50.732 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:50.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:50.733 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:50.733 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:50.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:50.734 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-snaptrim-stats 2026-02-13T07:05:50.764 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:50.764 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:50.764 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:50.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.150893 2026-02-13T07:05:50.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:50.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:50.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:34: run: teardown td/test-snaptrim-stats 2026-02-13T07:05:50.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-snaptrim-stats 2026-02-13T07:05:50.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:50.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-snaptrim-stats KILL 2026-02-13T07:05:50.767 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:50.767 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:50.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:50.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:50.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:50.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:50.768 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:50.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:50.769 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:50.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:50.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:50.770 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:50.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:50.771 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:50.771 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:50.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:50.772 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:50.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:50.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-snaptrim-stats 2026-02-13T07:05:50.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:50.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:50.774 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:50.774 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.150893 2026-02-13T07:05:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:31: run: for func in $funcs 2026-02-13T07:05:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:32: run: setup td/test-snaptrim-stats 2026-02-13T07:05:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-snaptrim-stats 2026-02-13T07:05:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-snaptrim-stats 2026-02-13T07:05:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-snaptrim-stats 2026-02-13T07:05:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:50.776 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-snaptrim-stats KILL 2026-02-13T07:05:50.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:50.777 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:50.777 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:50.777 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:50.777 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:50.777 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:50.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:50.778 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:50.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:50.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:50.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:50.779 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:50.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:50.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:50.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:50.781 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:50.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:50.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:50.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-snaptrim-stats 2026-02-13T07:05:50.783 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:50.783 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:50.783 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:50.783 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.150893 2026-02-13T07:05:50.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:50.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:50.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-snaptrim-stats 2026-02-13T07:05:50.785 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:05:50.785 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:50.785 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:50.786 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.150893 2026-02-13T07:05:50.786 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-snaptrim-stats 1' TERM HUP INT 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:33: run: TEST_snaptrim_stats_multiple_snaps td/test-snaptrim-stats 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:107: TEST_snaptrim_stats_multiple_snaps: local dir=td/test-snaptrim-stats 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:108: TEST_snaptrim_stats_multiple_snaps: local poolname=test 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:109: TEST_snaptrim_stats_multiple_snaps: local OSDS=3 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:110: TEST_snaptrim_stats_multiple_snaps: local PGNUM=8 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:111: TEST_snaptrim_stats_multiple_snaps: local PGPNUM=8 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:112: TEST_snaptrim_stats_multiple_snaps: local objects=10 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:113: TEST_snaptrim_stats_multiple_snaps: local WAIT_FOR_UPDATE=10 2026-02-13T07:05:50.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:115: TEST_snaptrim_stats_multiple_snaps: setup td/test-snaptrim-stats 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-snaptrim-stats 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-snaptrim-stats 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-snaptrim-stats 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-snaptrim-stats KILL 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:05:50.790 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:05:50.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:05:50.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:05:50.791 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:05:50.791 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:05:50.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:05:50.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:05:50.792 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:05:50.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:50.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:05:50.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:05:50.793 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:05:50.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:05:50.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:05:50.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-snaptrim-stats 2026-02-13T07:05:50.796 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:05:50.796 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:50.796 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:50.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.150893 2026-02-13T07:05:50.797 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:05:50.797 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:05:50.797 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-snaptrim-stats 2026-02-13T07:05:50.798 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:05:50.798 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:50.798 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:50.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.150893 2026-02-13T07:05:50.799 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:05:50.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:05:50.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:05:50.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-snaptrim-stats 1' TERM HUP INT 2026-02-13T07:05:50.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:116: TEST_snaptrim_stats_multiple_snaps: run_mon td/test-snaptrim-stats a --osd_pool_default_size=3 2026-02-13T07:05:50.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-snaptrim-stats 2026-02-13T07:05:50.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:05:50.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:05:50.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:05:50.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-snaptrim-stats/a 2026-02-13T07:05:50.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-snaptrim-stats/a --run-dir=td/test-snaptrim-stats --osd_pool_default_size=3 2026-02-13T07:05:50.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:05:50.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:50.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:50.820 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:50.820 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:50.820 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:50.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:50.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-snaptrim-stats/a '--log-file=td/test-snaptrim-stats/$name.log' '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --mon-cluster-log-file=td/test-snaptrim-stats/log --run-dir=td/test-snaptrim-stats '--pid-file=td/test-snaptrim-stats/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-13T07:05:50.847 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:05:50.847 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:05:50.848 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:05:50.848 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:05:50.848 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:05:50.849 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:05:50.849 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:05:50.849 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:05:50.849 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:05:50.849 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:05:50.849 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:50.850 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:50.850 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.150893/ceph-mon.a.asok 2026-02-13T07:05:50.850 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:05:50.850 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.150893/ceph-mon.a.asok config get fsid 2026-02-13T07:05:50.893 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:05:50.893 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:05:50.893 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:05:50.893 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:05:50.894 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:05:50.895 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:05:50.895 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:05:50.895 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:05:50.895 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:05:50.895 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:50.895 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:50.895 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.150893/ceph-mon.a.asok 2026-02-13T07:05:50.895 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:05:50.895 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.150893/ceph-mon.a.asok config get mon_host 2026-02-13T07:05:50.937 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:117: TEST_snaptrim_stats_multiple_snaps: run_mgr td/test-snaptrim-stats x 2026-02-13T07:05:50.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-snaptrim-stats 2026-02-13T07:05:50.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:05:50.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:05:50.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:05:50.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-snaptrim-stats/x 2026-02-13T07:05:50.938 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:05:51.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:05:51.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:51.038 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:51.038 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:51.039 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:51.039 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:51.039 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:51.039 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:05:51.039 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-snaptrim-stats/x '--log-file=td/test-snaptrim-stats/$name.log' '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --run-dir=td/test-snaptrim-stats '--pid-file=td/test-snaptrim-stats/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:05:51.055 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:118: TEST_snaptrim_stats_multiple_snaps: expr 3 - 1 2026-02-13T07:05:51.057 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:118: TEST_snaptrim_stats_multiple_snaps: seq 0 2 2026-02-13T07:05:51.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:118: TEST_snaptrim_stats_multiple_snaps: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-13T07:05:51.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:120: TEST_snaptrim_stats_multiple_snaps: run_osd td/test-snaptrim-stats 0 --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:51.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-snaptrim-stats 2026-02-13T07:05:51.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:05:51.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:05:51.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:05:51.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-snaptrim-stats/0 2026-02-13T07:05:51.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 ' 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-snaptrim-stats/0' 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-snaptrim-stats/0/journal' 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-snaptrim-stats' 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:51.062 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-snaptrim-stats/$name.log' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-snaptrim-stats/$name.pid' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:05:51.063 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:05:51.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:05:51.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:51.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-snaptrim-stats/0 2026-02-13T07:05:51.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:05:51.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9669b7b3-22fb-4dad-a1f9-073e29f4c310 2026-02-13T07:05:51.064 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9669b7b3-22fb-4dad-a1f9-073e29f4c310' 2026-02-13T07:05:51.064 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 9669b7b3-22fb-4dad-a1f9-073e29f4c310 2026-02-13T07:05:51.065 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:05:51.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBPzY5ptME6BBAA3MGISYoqgagqzm9Yo3n6jw== 2026-02-13T07:05:51.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBPzY5ptME6BBAA3MGISYoqgagqzm9Yo3n6jw=="}' 2026-02-13T07:05:51.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9669b7b3-22fb-4dad-a1f9-073e29f4c310 -i td/test-snaptrim-stats/0/new.json 2026-02-13T07:05:51.176 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:51.183 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-snaptrim-stats/0/new.json 2026-02-13T07:05:51.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/0 --osd-journal=td/test-snaptrim-stats/0/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --mkfs --key AQBPzY5ptME6BBAA3MGISYoqgagqzm9Yo3n6jw== --osd-uuid 9669b7b3-22fb-4dad-a1f9-073e29f4c310 2026-02-13T07:05:51.202 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:51.199+0000 7fc922383900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:51.204 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:51.201+0000 7fc922383900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:51.205 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:51.202+0000 7fc922383900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:51.205 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:51.202+0000 7fc922383900 -1 bdev(0x55ba27056c00 td/test-snaptrim-stats/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:05:51.205 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:51.202+0000 7fc922383900 -1 bluestore(td/test-snaptrim-stats/0) _read_fsid unparsable uuid 2026-02-13T07:05:51.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-snaptrim-stats/0/keyring 2026-02-13T07:05:51.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:05:51.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:05:51.682 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:05:51.682 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-snaptrim-stats/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:05:51.784 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:05:51.785 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:05:51.785 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/0 --osd-journal=td/test-snaptrim-stats/0/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:51.786 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:05:51.786 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:05:51.786 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:05:51.802 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:51.799+0000 7f514c40f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:51.803 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:51.800+0000 7f514c40f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:51.804 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:51.801+0000 7f514c40f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:51.883 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:05:51.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:05:51.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:05:51.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:05:51.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:05:51.884 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:51.884 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:51.885 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:05:51.885 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:51.885 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:51.976 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:51.973+0000 7f514c40f900 -1 Falling back to public interface 2026-02-13T07:05:51.981 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:52.421 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:52.419+0000 7f514c40f900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:05:52.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:52.982 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:52.982 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:52.983 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:05:52.983 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:52.983 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:53.085 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:54.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:54.087 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:54.087 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:05:54.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-13T07:05:54.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:54.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:05:54.314 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2143360381,v1:127.0.0.1:6801/2143360381] [v2:127.0.0.1:6802/2143360381,v1:127.0.0.1:6803/2143360381] exists,up 9669b7b3-22fb-4dad-a1f9-073e29f4c310 2026-02-13T07:05:54.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:05:54.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:05:54.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:05:54.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:118: TEST_snaptrim_stats_multiple_snaps: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-13T07:05:54.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:120: TEST_snaptrim_stats_multiple_snaps: run_osd td/test-snaptrim-stats 1 --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:54.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-snaptrim-stats 2026-02-13T07:05:54.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:05:54.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:05:54.318 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-snaptrim-stats/1 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 ' 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-snaptrim-stats/1' 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-snaptrim-stats/1/journal' 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-snaptrim-stats' 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:05:54.319 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-snaptrim-stats/$name.log' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-snaptrim-stats/$name.pid' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:05:54.320 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:05:54.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:05:54.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:05:54.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:05:54.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:54.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-snaptrim-stats/1 2026-02-13T07:05:54.321 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:05:54.321 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b1fcf137-ef77-4dd9-a224-4888d6760a2b 2026-02-13T07:05:54.321 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 b1fcf137-ef77-4dd9-a224-4888d6760a2b 2026-02-13T07:05:54.322 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b1fcf137-ef77-4dd9-a224-4888d6760a2b' 2026-02-13T07:05:54.322 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:05:54.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBSzY5pUZiBExAAdzNh3Dv0AYg9zorpYkOzAA== 2026-02-13T07:05:54.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBSzY5pUZiBExAAdzNh3Dv0AYg9zorpYkOzAA=="}' 2026-02-13T07:05:54.329 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b1fcf137-ef77-4dd9-a224-4888d6760a2b -i td/test-snaptrim-stats/1/new.json 2026-02-13T07:05:54.557 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:54.565 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-snaptrim-stats/1/new.json 2026-02-13T07:05:54.566 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/1 --osd-journal=td/test-snaptrim-stats/1/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --mkfs --key AQBSzY5pUZiBExAAdzNh3Dv0AYg9zorpYkOzAA== --osd-uuid b1fcf137-ef77-4dd9-a224-4888d6760a2b 2026-02-13T07:05:54.584 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:54.581+0000 7fbfa9ee0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:54.585 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:54.582+0000 7fbfa9ee0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:54.586 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:54.583+0000 7fbfa9ee0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:54.587 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:54.584+0000 7fbfa9ee0900 -1 bdev(0x557413fcec00 td/test-snaptrim-stats/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:05:54.587 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:54.584+0000 7fbfa9ee0900 -1 bluestore(td/test-snaptrim-stats/1) _read_fsid unparsable uuid 2026-02-13T07:05:54.907 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-snaptrim-stats/1/keyring 2026-02-13T07:05:54.907 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:05:54.908 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:05:54.909 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:05:54.909 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-snaptrim-stats/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:05:55.227 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:05:55.228 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:05:55.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/1 --osd-journal=td/test-snaptrim-stats/1/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:55.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:05:55.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:05:55.229 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:05:55.244 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:55.241+0000 7f27a9cba900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:55.245 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:55.242+0000 7f27a9cba900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:55.246 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:55.243+0000 7f27a9cba900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:55.396 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:55.392+0000 7f27a9cba900 -1 Falling back to public interface 2026-02-13T07:05:55.455 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:05:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:05:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:05:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:05:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:05:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:55.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:05:55.457 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:55.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:55.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:05:55.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:55.722 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:55.719+0000 7f27a9cba900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:05:56.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:56.691 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:56.691 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:56.692 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:05:56.692 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:56.692 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:05:56.915 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/538221959,v1:127.0.0.1:6811/538221959] [v2:127.0.0.1:6812/538221959,v1:127.0.0.1:6813/538221959] exists,up b1fcf137-ef77-4dd9-a224-4888d6760a2b 2026-02-13T07:05:56.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:05:56.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:05:56.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:05:56.919 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:118: TEST_snaptrim_stats_multiple_snaps: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:120: TEST_snaptrim_stats_multiple_snaps: run_osd td/test-snaptrim-stats 2 --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-snaptrim-stats 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-snaptrim-stats/2 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 ' 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-snaptrim-stats/2' 2026-02-13T07:05:56.920 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-snaptrim-stats/2/journal' 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-snaptrim-stats' 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:05:56.921 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-snaptrim-stats/$name.log' 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-snaptrim-stats/$name.pid' 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-snaptrim-stats/2 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:05:56.922 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0b71e42c-e77f-49d3-b002-ba5333fff1ea 2026-02-13T07:05:56.923 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 0b71e42c-e77f-49d3-b002-ba5333fff1ea' 2026-02-13T07:05:56.923 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 0b71e42c-e77f-49d3-b002-ba5333fff1ea 2026-02-13T07:05:56.923 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:05:56.931 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBUzY5peL5aNxAACPeyNwFBahkUYTvrm0jkmw== 2026-02-13T07:05:56.931 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBUzY5peL5aNxAACPeyNwFBahkUYTvrm0jkmw=="}' 2026-02-13T07:05:56.931 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0b71e42c-e77f-49d3-b002-ba5333fff1ea -i td/test-snaptrim-stats/2/new.json 2026-02-13T07:05:57.156 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:05:57.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-snaptrim-stats/2/new.json 2026-02-13T07:05:57.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/2 --osd-journal=td/test-snaptrim-stats/2/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --mkfs --key AQBUzY5peL5aNxAACPeyNwFBahkUYTvrm0jkmw== --osd-uuid 0b71e42c-e77f-49d3-b002-ba5333fff1ea 2026-02-13T07:05:57.181 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:57.178+0000 7f6d7c410900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:57.182 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:57.179+0000 7f6d7c410900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:57.183 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:57.180+0000 7f6d7c410900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:57.184 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:57.181+0000 7f6d7c410900 -1 bdev(0x55a0f3c58c00 td/test-snaptrim-stats/2/block) open stat got: (1) Operation not permitted 2026-02-13T07:05:57.184 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:57.181+0000 7f6d7c410900 -1 bluestore(td/test-snaptrim-stats/2) _read_fsid unparsable uuid 2026-02-13T07:05:57.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-snaptrim-stats/2/keyring 2026-02-13T07:05:57.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:05:57.514 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T07:05:57.514 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T07:05:57.514 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-snaptrim-stats/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:05:57.834 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T07:05:57.834 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:05:57.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=a61dcc71-d299-4239-925a-1b0dd6b7cfd1 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-snaptrim-stats/2 --osd-journal=td/test-snaptrim-stats/2/journal --chdir= --run-dir=td/test-snaptrim-stats '--admin-socket=/tmp/ceph-asok.150893/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-snaptrim-stats/$name.log' '--pid-file=td/test-snaptrim-stats/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off 2026-02-13T07:05:57.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:05:57.835 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:05:57.836 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:05:57.850 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:57.847+0000 7f02e8320900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:57.851 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:57.848+0000 7f02e8320900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:57.852 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:57.849+0000 7f02e8320900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:05:57.967 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:57.965+0000 7f02e8320900 -1 Falling back to public interface 2026-02-13T07:05:58.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T07:05:58.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:05:58.060 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:05:58.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:05:58.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:05:58.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:58.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:05:58.061 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:05:58.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:58.062 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:05:58.265 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:05:58.262+0000 7f02e8320900 -1 osd.2 0 log_to_monitors true 2026-02-13T07:05:58.286 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:05:59.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:05:59.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:05:59.287 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:05:59.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:05:59.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:05:59.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:05:59.514 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2734849640,v1:127.0.0.1:6819/2734849640] [v2:127.0.0.1:6820/2734849640,v1:127.0.0.1:6821/2734849640] exists,up 0b71e42c-e77f-49d3-b002-ba5333fff1ea 2026-02-13T07:05:59.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:05:59.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:05:59.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:05:59.515 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:124: TEST_snaptrim_stats_multiple_snaps: ceph osd set noscrub 2026-02-13T07:05:59.798 INFO:tasks.workunit.client.0.trial040.stderr:noscrub is set 2026-02-13T07:05:59.808 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:125: TEST_snaptrim_stats_multiple_snaps: ceph osd set nodeep-scrub 2026-02-13T07:06:00.100 INFO:tasks.workunit.client.0.trial040.stderr:nodeep-scrub is set 2026-02-13T07:06:00.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:128: TEST_snaptrim_stats_multiple_snaps: create_pool test 8 8 2026-02-13T07:06:00.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 8 8 2026-02-13T07:06:00.366 INFO:tasks.workunit.client.0.trial040.stderr:pool 'test' already exists 2026-02-13T07:06:00.374 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T07:06:01.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:129: TEST_snaptrim_stats_multiple_snaps: wait_for_clean 2026-02-13T07:06:01.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:06:01.376 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:06:01.377 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:06:01.377 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T07:06:01.378 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:06:01.378 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:06:01.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:06:01.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:06:01.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:06:01.462 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:06:01.462 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:06:01.462 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:06:01.462 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:06:01.463 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:06:01.463 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:06:01.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T07:06:01.693 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T07:06:01.693 INFO:tasks.workunit.client.0.trial040.stderr:2' 2026-02-13T07:06:01.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:06:01.693 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:06:01.693 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:06:01.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T07:06:01.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T07:06:01.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T07:06:01.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:06:01.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T07:06:01.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-13T07:06:01.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-13T07:06:01.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672963' 2026-02-13T07:06:01.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:06:01.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T07:06:01.886 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509444 2026-02-13T07:06:01.886 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509444 2026-02-13T07:06:01.886 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672963 2-64424509444' 2026-02-13T07:06:01.886 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:06:01.887 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T07:06:01.887 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:06:01.887 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:06:01.888 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-13T07:06:01.888 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:06:01.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-13T07:06:01.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T07:06:01.890 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836484 2026-02-13T07:06:01.890 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:06:02.116 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T07:06:02.117 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T07:06:03.118 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T07:06:03.118 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:06:03.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-13T07:06:03.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:06:03.348 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-13T07:06:03.348 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:06:03.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T07:06:03.349 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672963 2026-02-13T07:06:03.349 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:06:03.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672963 2026-02-13T07:06:03.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-13T07:06:03.351 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672963 2026-02-13T07:06:03.351 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T07:06:03.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672963 2026-02-13T07:06:03.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:06:03.576 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509444 2026-02-13T07:06:03.576 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:06:03.576 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T07:06:03.577 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509444 2026-02-13T07:06:03.577 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:06:03.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509444 2026-02-13T07:06:03.579 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-13T07:06:03.579 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 64424509444 2026-02-13T07:06:03.579 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T07:06:03.803 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509444 -lt 64424509444 2026-02-13T07:06:03.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:06:03.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:06:03.804 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:06:04.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 8 == 0 2026-02-13T07:06:04.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:06:04.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:06:04.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:06:04.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:06:04.133 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:06:04.134 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:06:04.134 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:06:04.354 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=8 2026-02-13T07:06:04.355 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:06:04.355 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:06:04.356 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:06:04.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 = 8 2026-02-13T07:06:04.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T07:06:04.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T07:06:04.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:130: TEST_snaptrim_stats_multiple_snaps: ceph osd dump 2026-02-13T07:06:04.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:130: TEST_snaptrim_stats_multiple_snaps: grep '^pool.*['\'']test['\'']' 2026-02-13T07:06:04.666 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:130: TEST_snaptrim_stats_multiple_snaps: awk '{ print $2 }' 2026-02-13T07:06:04.889 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:130: TEST_snaptrim_stats_multiple_snaps: poolid=1 2026-02-13T07:06:04.890 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:133: TEST_snaptrim_stats_multiple_snaps: local TESTDATA=testdata.0 2026-02-13T07:06:04.890 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:134: TEST_snaptrim_stats_multiple_snaps: dd if=/dev/urandom of=testdata.0 bs=4096 count=1 2026-02-13T07:06:04.890 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records in 2026-02-13T07:06:04.891 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records out 2026-02-13T07:06:04.891 INFO:tasks.workunit.client.0.trial040.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 4.0245e-05 s, 102 MB/s 2026-02-13T07:06:04.891 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: seq 1 10 2026-02-13T07:06:04.891 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:04.892 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj1 testdata.0 2026-02-13T07:06:04.910 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:04.910 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj2 testdata.0 2026-02-13T07:06:04.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:04.928 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj3 testdata.0 2026-02-13T07:06:04.945 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:04.945 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj4 testdata.0 2026-02-13T07:06:04.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:04.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj5 testdata.0 2026-02-13T07:06:04.980 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:04.980 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj6 testdata.0 2026-02-13T07:06:04.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:04.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj7 testdata.0 2026-02-13T07:06:05.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj8 testdata.0 2026-02-13T07:06:05.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.032 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj9 testdata.0 2026-02-13T07:06:05.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:135: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.049 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:137: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj10 testdata.0 2026-02-13T07:06:05.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:139: TEST_snaptrim_stats_multiple_snaps: rm -f testdata.0 2026-02-13T07:06:05.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:142: TEST_snaptrim_stats_multiple_snaps: NUMSNAPS=2 2026-02-13T07:06:05.067 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:143: TEST_snaptrim_stats_multiple_snaps: seq 1 2 2026-02-13T07:06:05.068 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:143: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $NUMSNAPS` 2026-02-13T07:06:05.068 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:145: TEST_snaptrim_stats_multiple_snaps: rados -p test mksnap snap1 2026-02-13T07:06:05.175 INFO:tasks.workunit.client.0.trial040.stdout:created pool test snap snap1 2026-02-13T07:06:05.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:146: TEST_snaptrim_stats_multiple_snaps: TESTDATA=testdata.1 2026-02-13T07:06:05.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:147: TEST_snaptrim_stats_multiple_snaps: dd if=/dev/urandom of=testdata.1 bs=4096 count=1 2026-02-13T07:06:05.178 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records in 2026-02-13T07:06:05.178 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records out 2026-02-13T07:06:05.178 INFO:tasks.workunit.client.0.trial040.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 4.4753e-05 s, 91.5 MB/s 2026-02-13T07:06:05.179 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: seq 1 10 2026-02-13T07:06:05.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.179 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj1 testdata.1 2026-02-13T07:06:05.198 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.198 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj2 testdata.1 2026-02-13T07:06:05.217 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.217 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj3 testdata.1 2026-02-13T07:06:05.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.235 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj4 testdata.1 2026-02-13T07:06:05.253 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.253 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj5 testdata.1 2026-02-13T07:06:05.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj6 testdata.1 2026-02-13T07:06:05.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.289 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj7 testdata.1 2026-02-13T07:06:05.306 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj8 testdata.1 2026-02-13T07:06:05.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.325 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj9 testdata.1 2026-02-13T07:06:05.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj10 testdata.1 2026-02-13T07:06:05.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:152: TEST_snaptrim_stats_multiple_snaps: rm -f testdata.1 2026-02-13T07:06:05.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:143: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $NUMSNAPS` 2026-02-13T07:06:05.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:145: TEST_snaptrim_stats_multiple_snaps: rados -p test mksnap snap2 2026-02-13T07:06:05.430 INFO:tasks.workunit.client.0.trial040.stdout:created pool test snap snap2 2026-02-13T07:06:05.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:146: TEST_snaptrim_stats_multiple_snaps: TESTDATA=testdata.2 2026-02-13T07:06:05.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:147: TEST_snaptrim_stats_multiple_snaps: dd if=/dev/urandom of=testdata.2 bs=4096 count=1 2026-02-13T07:06:05.432 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records in 2026-02-13T07:06:05.432 INFO:tasks.workunit.client.0.trial040.stderr:1+0 records out 2026-02-13T07:06:05.432 INFO:tasks.workunit.client.0.trial040.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 4.3061e-05 s, 95.1 MB/s 2026-02-13T07:06:05.433 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: seq 1 10 2026-02-13T07:06:05.433 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.433 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj1 testdata.2 2026-02-13T07:06:05.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.452 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj2 testdata.2 2026-02-13T07:06:05.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj3 testdata.2 2026-02-13T07:06:05.488 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.488 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj4 testdata.2 2026-02-13T07:06:05.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.506 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj5 testdata.2 2026-02-13T07:06:05.523 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.523 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj6 testdata.2 2026-02-13T07:06:05.540 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.540 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj7 testdata.2 2026-02-13T07:06:05.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj8 testdata.2 2026-02-13T07:06:05.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.576 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj9 testdata.2 2026-02-13T07:06:05.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:148: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $objects` 2026-02-13T07:06:05.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:150: TEST_snaptrim_stats_multiple_snaps: rados -p test put obj10 testdata.2 2026-02-13T07:06:05.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:152: TEST_snaptrim_stats_multiple_snaps: rm -f testdata.2 2026-02-13T07:06:05.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:156: TEST_snaptrim_stats_multiple_snaps: local total_objects_trimmed=0 2026-02-13T07:06:05.612 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:157: TEST_snaptrim_stats_multiple_snaps: seq 1 2 2026-02-13T07:06:05.613 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:157: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $NUMSNAPS` 2026-02-13T07:06:05.613 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:159: TEST_snaptrim_stats_multiple_snaps: rados -p test rmsnap snap1 2026-02-13T07:06:05.631 INFO:tasks.workunit.client.0.trial040.stdout:removed pool test snap snap1 2026-02-13T07:06:05.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:162: TEST_snaptrim_stats_multiple_snaps: wait_for_clean 2026-02-13T07:06:05.632 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:06:05.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:06:05.633 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:06:05.633 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T07:06:05.634 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:06:05.634 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:06:05.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:06:05.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:06:05.634 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:06:05.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:06:05.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:06:05.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:06:05.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:06:05.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:06:05.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:06:05.940 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T07:06:05.940 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T07:06:05.940 INFO:tasks.workunit.client.0.trial040.stderr:2' 2026-02-13T07:06:05.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:06:05.941 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:06:05.941 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:06:06.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-13T07:06:06.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-13T07:06:06.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-13T07:06:06.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:06:06.006 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T07:06:06.068 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672966 2026-02-13T07:06:06.068 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672966 2026-02-13T07:06:06.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672966' 2026-02-13T07:06:06.069 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:06:06.069 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T07:06:06.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509447 2026-02-13T07:06:06.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509447 2026-02-13T07:06:06.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672966 2-64424509447' 2026-02-13T07:06:06.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:06:06.135 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-13T07:06:06.135 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:06:06.135 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:06:06.136 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-13T07:06:06.136 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:06:06.137 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-13T07:06:06.137 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-13T07:06:06.137 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836487 2026-02-13T07:06:06.138 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:06:06.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-13T07:06:06.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T07:06:07.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T07:06:07.364 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:06:07.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-13T07:06:07.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:06:07.592 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672966 2026-02-13T07:06:07.592 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:06:07.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T07:06:07.593 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672966 2026-02-13T07:06:07.593 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:06:07.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672966 2026-02-13T07:06:07.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-13T07:06:07.594 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672966 2026-02-13T07:06:07.595 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T07:06:07.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672967 -lt 42949672966 2026-02-13T07:06:07.817 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:06:07.818 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509447 2026-02-13T07:06:07.818 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:06:07.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T07:06:07.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509447 2026-02-13T07:06:07.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:06:07.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509447 2026-02-13T07:06:07.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2026-02-13T07:06:07.821 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 64424509447 2026-02-13T07:06:07.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T07:06:08.044 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509447 -lt 64424509447 2026-02-13T07:06:08.044 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:06:08.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:06:08.045 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:06:08.362 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 8 == 0 2026-02-13T07:06:08.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:06:08.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:06:08.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:06:08.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:06:08.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:06:08.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:06:08.363 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:06:08.586 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=8 2026-02-13T07:06:08.586 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:06:08.587 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:06:08.587 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:06:08.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 = 8 2026-02-13T07:06:08.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T07:06:08.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T07:06:08.898 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:163: TEST_snaptrim_stats_multiple_snaps: sleep 10 2026-02-13T07:06:18.899 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:164: TEST_snaptrim_stats_multiple_snaps: local objects_trimmed=0 2026-02-13T07:06:18.899 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:165: TEST_snaptrim_stats_multiple_snaps: local snaptrim_duration_total=0.0 2026-02-13T07:06:18.900 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: expr 8 - 1 2026-02-13T07:06:18.901 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: seq 0 7 2026-02-13T07:06:18.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:18.902 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.0 2026-02-13T07:06:18.904 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.0 query 2026-02-13T07:06:18.904 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:18.970 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 0 + 2 2026-02-13T07:06:18.971 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=2 2026-02-13T07:06:18.972 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:18.973 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.0 query 2026-02-13T07:06:18.973 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:19.040 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo 0.0 + 0.0012466790000000001 2026-02-13T07:06:19.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.0012466790000000001 2026-02-13T07:06:19.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:19.041 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.1 2026-02-13T07:06:19.041 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.1 query 2026-02-13T07:06:19.042 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:19.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 2 + 2 2026-02-13T07:06:19.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=4 2026-02-13T07:06:19.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:19.110 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.1 query 2026-02-13T07:06:19.111 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:19.176 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .0012466790000000001 + 0.0016222280000000001 2026-02-13T07:06:19.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.0028689070000000002 2026-02-13T07:06:19.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:19.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.2 2026-02-13T07:06:19.177 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.2 query 2026-02-13T07:06:19.177 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:19.244 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 4 + 0 2026-02-13T07:06:19.245 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=4 2026-02-13T07:06:19.246 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:19.247 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.2 query 2026-02-13T07:06:19.247 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:19.312 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .0028689070000000002 + 0.00026409099999999999 2026-02-13T07:06:19.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00313299800000000019 2026-02-13T07:06:19.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:19.312 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.3 2026-02-13T07:06:19.313 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.3 query 2026-02-13T07:06:19.313 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:19.378 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 4 + 2 2026-02-13T07:06:19.379 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=6 2026-02-13T07:06:19.380 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:19.380 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.3 query 2026-02-13T07:06:19.381 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:19.445 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00313299800000000019 + 0.0020313800000000002 2026-02-13T07:06:19.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00516437800000000039 2026-02-13T07:06:19.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:19.446 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.4 2026-02-13T07:06:19.447 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.4 query 2026-02-13T07:06:19.447 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:19.516 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 6 + 1 2026-02-13T07:06:19.516 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=7 2026-02-13T07:06:19.517 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:19.518 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.4 query 2026-02-13T07:06:19.518 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:19.584 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00516437800000000039 + 0.00099540200000000005 2026-02-13T07:06:19.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00615978000000000044 2026-02-13T07:06:19.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:19.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.5 2026-02-13T07:06:19.586 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.5 query 2026-02-13T07:06:19.586 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:19.651 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 7 + 1 2026-02-13T07:06:19.652 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=8 2026-02-13T07:06:19.653 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:19.653 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.5 query 2026-02-13T07:06:19.653 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:19.722 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00615978000000000044 + 0.00139204 2026-02-13T07:06:19.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00755182000000000044 2026-02-13T07:06:19.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:19.723 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.6 2026-02-13T07:06:19.723 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.6 query 2026-02-13T07:06:19.724 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:19.788 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 8 + 0 2026-02-13T07:06:19.789 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=8 2026-02-13T07:06:19.790 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:19.790 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.6 query 2026-02-13T07:06:19.791 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:19.857 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00755182000000000044 + 0.001523564 2026-02-13T07:06:19.858 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00907538400000000044 2026-02-13T07:06:19.858 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:19.858 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.7 2026-02-13T07:06:19.859 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.7 query 2026-02-13T07:06:19.859 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:19.925 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 8 + 2 2026-02-13T07:06:19.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=10 2026-02-13T07:06:19.927 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:19.927 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.7 query 2026-02-13T07:06:19.928 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:19.993 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00907538400000000044 + 0.0024957930000000001 2026-02-13T07:06:19.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.01157117700000000054 2026-02-13T07:06:19.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:174: TEST_snaptrim_stats_multiple_snaps: test 10 -eq 10 2026-02-13T07:06:19.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:175: TEST_snaptrim_stats_multiple_snaps: echo '.01157117700000000054 > 0.0' 2026-02-13T07:06:19.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:175: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:19.995 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:06:19.995 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:176: TEST_snaptrim_stats_multiple_snaps: expr 0 + 10 2026-02-13T07:06:19.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:176: TEST_snaptrim_stats_multiple_snaps: total_objects_trimmed=10 2026-02-13T07:06:19.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:157: TEST_snaptrim_stats_multiple_snaps: for i in `seq 1 $NUMSNAPS` 2026-02-13T07:06:19.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:159: TEST_snaptrim_stats_multiple_snaps: rados -p test rmsnap snap2 2026-02-13T07:06:20.064 INFO:tasks.workunit.client.0.trial040.stdout:removed pool test snap snap2 2026-02-13T07:06:20.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:162: TEST_snaptrim_stats_multiple_snaps: wait_for_clean 2026-02-13T07:06:20.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T07:06:20.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T07:06:20.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T07:06:20.068 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T07:06:20.068 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T07:06:20.068 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T07:06:20.069 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T07:06:20.069 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T07:06:20.069 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T07:06:20.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T07:06:20.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T07:06:20.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T07:06:20.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T07:06:20.150 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T07:06:20.150 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T07:06:20.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T07:06:20.385 INFO:tasks.workunit.client.0.trial040.stderr:1 2026-02-13T07:06:20.385 INFO:tasks.workunit.client.0.trial040.stderr:2' 2026-02-13T07:06:20.386 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T07:06:20.386 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:06:20.386 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-13T07:06:20.448 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836492 2026-02-13T07:06:20.449 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836492 2026-02-13T07:06:20.449 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492' 2026-02-13T07:06:20.449 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:06:20.449 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-13T07:06:20.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672971 2026-02-13T07:06:20.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672971 2026-02-13T07:06:20.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672971' 2026-02-13T07:06:20.513 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T07:06:20.513 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-13T07:06:20.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509452 2026-02-13T07:06:20.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509452 2026-02-13T07:06:20.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-64424509452' 2026-02-13T07:06:20.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:06:20.575 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836492 2026-02-13T07:06:20.576 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:06:20.576 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-13T07:06:20.577 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836492 2026-02-13T07:06:20.577 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:06:20.577 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836492 2026-02-13T07:06:20.578 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.0 seq 21474836492 2026-02-13T07:06:20.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-13T07:06:20.578 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:06:20.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-13T07:06:20.800 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-13T07:06:21.801 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T07:06:21.802 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T07:06:22.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-13T07:06:22.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:06:22.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672971 2026-02-13T07:06:22.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:06:22.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-13T07:06:22.028 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672971 2026-02-13T07:06:22.029 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:06:22.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672971 2026-02-13T07:06:22.030 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.1 seq 42949672971 2026-02-13T07:06:22.030 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-13T07:06:22.030 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T07:06:22.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-13T07:06:22.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-13T07:06:22.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509452 2026-02-13T07:06:22.256 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-13T07:06:22.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-13T07:06:22.258 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509452 2026-02-13T07:06:22.258 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-13T07:06:22.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509452 2026-02-13T07:06:22.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509452' 2026-02-13T07:06:22.259 INFO:tasks.workunit.client.0.trial040.stdout:waiting osd.2 seq 64424509452 2026-02-13T07:06:22.260 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T07:06:22.487 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509452 -lt 64424509452 2026-02-13T07:06:22.488 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T07:06:22.488 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:06:22.488 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:06:22.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 8 == 0 2026-02-13T07:06:22.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T07:06:22.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T07:06:22.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T07:06:22.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T07:06:22.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T07:06:22.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T07:06:22.797 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T07:06:23.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=8 2026-02-13T07:06:23.018 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T07:06:23.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T07:06:23.019 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T07:06:23.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 = 8 2026-02-13T07:06:23.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T07:06:23.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T07:06:23.337 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:163: TEST_snaptrim_stats_multiple_snaps: sleep 10 2026-02-13T07:06:33.338 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:164: TEST_snaptrim_stats_multiple_snaps: local objects_trimmed=0 2026-02-13T07:06:33.338 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:165: TEST_snaptrim_stats_multiple_snaps: local snaptrim_duration_total=0.0 2026-02-13T07:06:33.338 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: expr 8 - 1 2026-02-13T07:06:33.340 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: seq 0 7 2026-02-13T07:06:33.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:33.341 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.0 2026-02-13T07:06:33.342 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.0 query 2026-02-13T07:06:33.342 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:33.407 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 0 + 2 2026-02-13T07:06:33.408 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=2 2026-02-13T07:06:33.409 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:33.410 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.0 query 2026-02-13T07:06:33.410 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:33.475 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo 0.0 + 0.001415714 2026-02-13T07:06:33.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.001415714 2026-02-13T07:06:33.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:33.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.1 2026-02-13T07:06:33.476 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.1 query 2026-02-13T07:06:33.476 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:33.543 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 2 + 2 2026-02-13T07:06:33.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=4 2026-02-13T07:06:33.545 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:33.545 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.1 query 2026-02-13T07:06:33.545 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:33.610 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .001415714 + 0.0015708720000000001 2026-02-13T07:06:33.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.0029865860000000001 2026-02-13T07:06:33.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:33.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.2 2026-02-13T07:06:33.612 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.2 query 2026-02-13T07:06:33.613 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:33.678 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 4 + 0 2026-02-13T07:06:33.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=4 2026-02-13T07:06:33.679 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:33.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.2 query 2026-02-13T07:06:33.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:33.747 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .0029865860000000001 + 0.00030478699999999999 2026-02-13T07:06:33.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00329137300000000009 2026-02-13T07:06:33.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:33.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.3 2026-02-13T07:06:33.749 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.3 query 2026-02-13T07:06:33.749 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:33.814 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 4 + 2 2026-02-13T07:06:33.815 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=6 2026-02-13T07:06:33.816 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:33.816 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.3 query 2026-02-13T07:06:33.817 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:33.881 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00329137300000000009 + 0.001329553 2026-02-13T07:06:33.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00462092600000000009 2026-02-13T07:06:33.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:33.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.4 2026-02-13T07:06:33.883 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.4 query 2026-02-13T07:06:33.883 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:33.950 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 6 + 1 2026-02-13T07:06:33.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=7 2026-02-13T07:06:33.951 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:33.952 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.4 query 2026-02-13T07:06:33.952 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:34.017 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00462092600000000009 + 0.0023467560000000002 2026-02-13T07:06:34.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00696768200000000029 2026-02-13T07:06:34.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:34.018 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.5 2026-02-13T07:06:34.020 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.5 query 2026-02-13T07:06:34.020 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:34.085 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 7 + 1 2026-02-13T07:06:34.086 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=8 2026-02-13T07:06:34.086 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:34.087 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.5 query 2026-02-13T07:06:34.087 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:34.154 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00696768200000000029 + 0.001278649 2026-02-13T07:06:34.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00824633100000000029 2026-02-13T07:06:34.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:34.155 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.6 2026-02-13T07:06:34.155 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.6 query 2026-02-13T07:06:34.155 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:34.221 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 8 + 0 2026-02-13T07:06:34.222 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=8 2026-02-13T07:06:34.223 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:34.224 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.6 query 2026-02-13T07:06:34.224 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:34.291 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00824633100000000029 + 0.0015025349999999999 2026-02-13T07:06:34.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.00974886600000000019 2026-02-13T07:06:34.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:166: TEST_snaptrim_stats_multiple_snaps: for i in $(seq 0 $(expr $PGNUM - 1)) 2026-02-13T07:06:34.292 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:168: TEST_snaptrim_stats_multiple_snaps: local pgid=1.7 2026-02-13T07:06:34.292 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.7 query 2026-02-13T07:06:34.293 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.objects_trimmed 2026-02-13T07:06:34.360 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: expr 8 + 2 2026-02-13T07:06:34.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:170: TEST_snaptrim_stats_multiple_snaps: objects_trimmed=10 2026-02-13T07:06:34.362 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:34.362 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: ceph pg 1.7 query 2026-02-13T07:06:34.362 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: jq .info.stats.snaptrim_duration 2026-02-13T07:06:34.430 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: echo .00974886600000000019 + 0.0023573259999999999 2026-02-13T07:06:34.430 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:172: TEST_snaptrim_stats_multiple_snaps: snaptrim_duration_total=.01210619200000000009 2026-02-13T07:06:34.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:174: TEST_snaptrim_stats_multiple_snaps: test 10 -eq 10 2026-02-13T07:06:34.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:175: TEST_snaptrim_stats_multiple_snaps: echo '.01210619200000000009 > 0.0' 2026-02-13T07:06:34.431 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:175: TEST_snaptrim_stats_multiple_snaps: bc 2026-02-13T07:06:34.431 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:06:34.432 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:176: TEST_snaptrim_stats_multiple_snaps: expr 10 + 10 2026-02-13T07:06:34.433 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:176: TEST_snaptrim_stats_multiple_snaps: total_objects_trimmed=20 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:179: TEST_snaptrim_stats_multiple_snaps: test 20 -eq 20 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:181: TEST_snaptrim_stats_multiple_snaps: teardown td/test-snaptrim-stats 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-snaptrim-stats 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-snaptrim-stats KILL 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:06:34.434 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:06:34.541 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:06:34.541 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:06:34.542 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:06:34.542 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:06:34.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:06:34.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:06:34.544 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:06:34.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:06:34.545 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:06:34.546 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.546 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:06:34.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:06:34.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-snaptrim-stats 2026-02-13T07:06:34.578 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:06:34.578 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.578 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:06:34.578 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.150893 2026-02-13T07:06:34.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:06:34.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:06:34.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-snaptrim-stats.sh:34: run: teardown td/test-snaptrim-stats 2026-02-13T07:06:34.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-snaptrim-stats 2026-02-13T07:06:34.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:06:34.580 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-snaptrim-stats KILL 2026-02-13T07:06:34.580 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:06:34.580 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:06:34.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:06:34.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:06:34.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:06:34.581 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:06:34.581 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:06:34.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:06:34.583 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:06:34.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:06:34.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:06:34.584 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:06:34.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:06:34.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:06:34.585 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.586 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:06:34.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:06:34.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-snaptrim-stats 2026-02-13T07:06:34.588 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:06:34.588 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.588 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:06:34.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.150893 2026-02-13T07:06:34.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:06:34.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:06:34.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-13T07:06:34.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/test-snaptrim-stats 0 2026-02-13T07:06:34.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-snaptrim-stats 2026-02-13T07:06:34.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T07:06:34.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-snaptrim-stats KILL 2026-02-13T07:06:34.590 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:06:34.590 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:06:34.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:06:34.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:06:34.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:06:34.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:06:34.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:06:34.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:06:34.592 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:06:34.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:06:34.593 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:06:34.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:06:34.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:06:34.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:06:34.595 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:06:34.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T07:06:34.597 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-snaptrim-stats 2026-02-13T07:06:34.598 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:06:34.598 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.598 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.150893 2026-02-13T07:06:34.598 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.150893 2026-02-13T07:06:34.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:06:34.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:06:34.600 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-13T07:06:34.601 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T07:06:34.601 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T07:06:34.627 INFO:tasks.workunit:Running workunit misc/ver-health.sh... 2026-02-13T07:06:34.627 DEBUG:teuthology.orchestra.run.trial040:workunit test misc/ver-health.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3b40640f18f2d34a761ee9b6dba2dfbb2a3118eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh 2026-02-13T07:06:34.690 INFO:tasks.workunit.client.0.trial040.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T07:06:34.695 INFO:tasks.workunit.client.0.trial040.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T07:06:34.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T07:06:34.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T07:06:34.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-13T07:06:34.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-13T07:06:34.695 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/ver-health 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:20: run: local dir=td/ver-health 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:21: run: shift 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:23: run: export CEPH_MON_A=127.0.0.1:7165 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:23: run: CEPH_MON_A=127.0.0.1:7165 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:24: run: export CEPH_MON_B=127.0.0.1:7166 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:24: run: CEPH_MON_B=127.0.0.1:7166 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:25: run: export CEPH_ARGS 2026-02-13T07:06:34.696 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:26: run: uuidgen 2026-02-13T07:06:34.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:26: run: CEPH_ARGS+='--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none ' 2026-02-13T07:06:34.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:27: run: CEPH_ARGS+='--mon-host= ' 2026-02-13T07:06:34.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:28: run: CEPH_ARGS+='--mon_health_to_clog_tick_interval=1.0 ' 2026-02-13T07:06:34.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:29: run: export 'ORIG_CEPH_ARGS=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 ' 2026-02-13T07:06:34.697 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:29: run: ORIG_CEPH_ARGS='--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 ' 2026-02-13T07:06:34.697 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:31: run: set 2026-02-13T07:06:34.697 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T07:06:34.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:31: run: local 'funcs=TEST_check_version_health_1 2026-02-13T07:06:34.699 INFO:tasks.workunit.client.0.trial040.stderr:TEST_check_version_health_2 2026-02-13T07:06:34.699 INFO:tasks.workunit.client.0.trial040.stderr:TEST_check_version_health_3' 2026-02-13T07:06:34.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:32: run: for func in $funcs 2026-02-13T07:06:34.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:33: run: setup td/ver-health 2026-02-13T07:06:34.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ver-health 2026-02-13T07:06:34.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ver-health 2026-02-13T07:06:34.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:06:34.699 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:06:34.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:06:34.700 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:06:34.700 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:06:34.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:06:34.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:06:34.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:06:34.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:06:34.700 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:06:34.700 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:06:34.701 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:06:34.701 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:06:34.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:06:34.702 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:06:34.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:06:34.702 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:06:34.702 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.703 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:06:34.703 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:06:34.703 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:06:34.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:06:34.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.704 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:06:34.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:06:34.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:06:34.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ver-health 2026-02-13T07:06:34.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:06:34.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.706 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.706 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.164153 2026-02-13T07:06:34.707 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:06:34.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T07:06:34.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T07:06:34.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:06:34.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ver-health 1' TERM HUP INT 2026-02-13T07:06:34.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:34: run: TEST_check_version_health_1 td/ver-health 2026-02-13T07:06:34.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:68: TEST_check_version_health_1: local dir=td/ver-health 2026-02-13T07:06:34.709 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:71: TEST_check_version_health_1: CEPH_ARGS='--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:73: TEST_check_version_health_1: setup td/ver-health 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ver-health 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ver-health 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:06:34.710 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:06:34.711 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:06:34.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:06:34.711 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:06:34.711 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:06:34.712 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:06:34.712 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:06:34.712 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:06:34.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:06:34.713 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:06:34.713 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:06:34.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:06:34.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:06:34.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:06:34.716 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:06:34.716 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.716 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.716 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:06:34.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:06:34.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:06:34.717 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ver-health 2026-02-13T07:06:34.718 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:06:34.718 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.718 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.718 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.164153 2026-02-13T07:06:34.719 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:06:34.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:06:34.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:06:34.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ver-health 1' TERM HUP INT 2026-02-13T07:06:34.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:76: TEST_check_version_health_1: run_mon td/ver-health a --public-addr=127.0.0.1:7165 --mon_warn_older_version_delay=0 2026-02-13T07:06:34.720 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ver-health 2026-02-13T07:06:34.721 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:06:34.721 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:06:34.721 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:06:34.721 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ver-health/a 2026-02-13T07:06:34.721 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ver-health/a --run-dir=td/ver-health --public-addr=127.0.0.1:7165 --mon_warn_older_version_delay=0 2026-02-13T07:06:34.742 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:06:34.742 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:06:34.742 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:06:34.742 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:06:34.742 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.742 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.742 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:34.742 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ver-health/a '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --mon-cluster-log-file=td/ver-health/log --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7165 --mon_warn_older_version_delay=0 2026-02-13T07:06:34.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:06:34.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:06:34.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:06:34.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:06:34.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:06:34.771 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:06:34.773 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:06:34.773 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:06:34.773 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:06:34.773 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:06:34.773 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.773 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.773 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.a.asok 2026-02-13T07:06:34.773 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:06:34.773 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.a.asok config get fsid 2026-02-13T07:06:34.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:06:34.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:06:34.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:06:34.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:06:34.820 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:06:34.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:06:34.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:06:34.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:06:34.821 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:06:34.821 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.821 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.a.asok 2026-02-13T07:06:34.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:06:34.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.a.asok config get mon_host 2026-02-13T07:06:34.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:77: TEST_check_version_health_1: run_mon td/ver-health b --public-addr=127.0.0.1:7166 --mon_warn_older_version_delay=0 2026-02-13T07:06:34.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ver-health 2026-02-13T07:06:34.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:06:34.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T07:06:34.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:06:34.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ver-health/b 2026-02-13T07:06:34.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ver-health/b --run-dir=td/ver-health --public-addr=127.0.0.1:7166 --mon_warn_older_version_delay=0 2026-02-13T07:06:34.884 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:06:34.884 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:06:34.884 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:06:34.884 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:06:34.884 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.884 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.885 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:34.885 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ver-health/b '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --mon-cluster-log-file=td/ver-health/log --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7166 --mon_warn_older_version_delay=0 2026-02-13T07:06:34.910 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:06:34.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T07:06:34.911 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:06:34.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T07:06:34.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:06:34.912 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:06:34.913 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T07:06:34.913 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T07:06:34.913 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T07:06:34.913 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:06:34.913 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.913 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.913 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.b.asok 2026-02-13T07:06:34.913 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:06:34.914 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.b.asok config get fsid 2026-02-13T07:06:34.956 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T07:06:34.956 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:06:34.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T07:06:34.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:06:34.957 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:06:34.958 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T07:06:34.958 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T07:06:34.958 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T07:06:34.958 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:06:34.958 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:34.958 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:34.958 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.b.asok 2026-02-13T07:06:34.958 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:06:34.959 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.b.asok config get mon_host 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:78: TEST_check_version_health_1: run_osd td/ver-health 0 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ver-health 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ver-health/0 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:06:35.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ver-health/0' 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ver-health/0/journal' 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:35.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:06:35.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:06:35.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ver-health/0 2026-02-13T07:06:35.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:06:35.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=01283a02-6bdb-4e4b-8e2b-135b45da8566 2026-02-13T07:06:35.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 01283a02-6bdb-4e4b-8e2b-135b45da8566' 2026-02-13T07:06:35.009 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 01283a02-6bdb-4e4b-8e2b-135b45da8566 2026-02-13T07:06:35.009 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:06:35.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB7zY5pkoDlABAANxI4xs2AYPrMzF0cbBP34w== 2026-02-13T07:06:35.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB7zY5pkoDlABAANxI4xs2AYPrMzF0cbBP34w=="}' 2026-02-13T07:06:35.017 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 01283a02-6bdb-4e4b-8e2b-135b45da8566 -i td/ver-health/0/new.json 2026-02-13T07:06:41.122 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:06:41.129 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ver-health/0/new.json 2026-02-13T07:06:41.130 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/0 --osd-journal=td/ver-health/0/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB7zY5pkoDlABAANxI4xs2AYPrMzF0cbBP34w== --osd-uuid 01283a02-6bdb-4e4b-8e2b-135b45da8566 2026-02-13T07:06:41.147 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:41.144+0000 7f12493b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:41.149 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:41.146+0000 7f12493b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:41.150 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:41.147+0000 7f12493b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:41.150 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:41.147+0000 7f12493b0900 -1 bdev(0x55c81b14ec00 td/ver-health/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:06:41.150 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:41.147+0000 7f12493b0900 -1 bluestore(td/ver-health/0) _read_fsid unparsable uuid 2026-02-13T07:06:41.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ver-health/0/keyring 2026-02-13T07:06:41.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:06:41.469 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:06:41.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:06:41.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ver-health/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:06:41.571 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:06:41.571 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:06:41.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/0 --osd-journal=td/ver-health/0/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:06:41.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:06:41.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:06:41.572 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:06:41.587 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:41.584+0000 7f5367f5b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:41.588 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:41.586+0000 7f5367f5b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:41.589 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:41.587+0000 7f5367f5b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:41.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:06:41.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:06:41.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:06:41.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:06:41.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:06:41.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:41.668 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:06:41.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:06:41.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:41.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:06:41.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:06:41.794 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:41.792+0000 7f5367f5b900 -1 Falling back to public interface 2026-02-13T07:06:42.062 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:42.059+0000 7f5367f5b900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:06:42.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:06:42.766 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:42.766 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:06:42.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:06:42.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:42.767 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:06:42.866 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/335935320,v1:127.0.0.1:6801/335935320] [v2:127.0.0.1:6802/335935320,v1:127.0.0.1:6803/335935320] exists,up 01283a02-6bdb-4e4b-8e2b-135b45da8566 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:79: TEST_check_version_health_1: run_osd td/ver-health 1 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ver-health 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ver-health/1 2026-02-13T07:06:42.870 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ver-health/1' 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ver-health/1/journal' 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:06:42.871 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:06:42.872 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:06:42.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:06:42.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:06:42.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ver-health/1 2026-02-13T07:06:42.873 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:06:42.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a235b734-ab7c-451b-8dfd-ed0349ad778e 2026-02-13T07:06:42.873 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 a235b734-ab7c-451b-8dfd-ed0349ad778e 2026-02-13T07:06:42.874 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a235b734-ab7c-451b-8dfd-ed0349ad778e' 2026-02-13T07:06:42.874 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:06:42.881 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCCzY5phZtuNBAAHM5Nga82pf91QTDlqzcc1A== 2026-02-13T07:06:42.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCCzY5phZtuNBAAHM5Nga82pf91QTDlqzcc1A=="}' 2026-02-13T07:06:42.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a235b734-ab7c-451b-8dfd-ed0349ad778e -i td/ver-health/1/new.json 2026-02-13T07:06:42.987 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:06:42.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ver-health/1/new.json 2026-02-13T07:06:42.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/1 --osd-journal=td/ver-health/1/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCCzY5phZtuNBAAHM5Nga82pf91QTDlqzcc1A== --osd-uuid a235b734-ab7c-451b-8dfd-ed0349ad778e 2026-02-13T07:06:43.010 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.008+0000 7fd568959900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:43.012 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.009+0000 7fd568959900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:43.013 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.010+0000 7fd568959900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:43.013 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.010+0000 7fd568959900 -1 bdev(0x561798d5cc00 td/ver-health/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:06:43.013 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.010+0000 7fd568959900 -1 bluestore(td/ver-health/1) _read_fsid unparsable uuid 2026-02-13T07:06:43.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ver-health/1/keyring 2026-02-13T07:06:43.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:06:43.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:06:43.348 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:06:43.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ver-health/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:06:43.450 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:06:43.450 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:06:43.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/1 --osd-journal=td/ver-health/1/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:06:43.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:06:43.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:06:43.451 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:06:43.467 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.464+0000 7f1b8eeaf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:43.468 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.465+0000 7f1b8eeaf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:43.469 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.466+0000 7f1b8eeaf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:43.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:06:43.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:06:43.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:06:43.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:06:43.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:06:43.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:43.549 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:06:43.549 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:06:43.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:43.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:06:43.645 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:06:43.648 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.646+0000 7f1b8eeaf900 -1 Falling back to public interface 2026-02-13T07:06:43.896 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:43.893+0000 7f1b8eeaf900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:06:44.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:06:44.647 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:44.647 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:06:44.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:06:44.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:44.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:06:44.749 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/4144127484,v1:127.0.0.1:6809/4144127484] [v2:127.0.0.1:6810/4144127484,v1:127.0.0.1:6811/4144127484] exists,up a235b734-ab7c-451b-8dfd-ed0349ad778e 2026-02-13T07:06:44.752 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:80: TEST_check_version_health_1: run_osd td/ver-health 2 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ver-health 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ver-health/2 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:06:44.753 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ver-health/2' 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ver-health/2/journal' 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:44.754 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:06:44.755 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:06:44.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ver-health/2 2026-02-13T07:06:44.756 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:06:44.756 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d3569347-f7c6-4f21-8726-e1cc18a8f406 2026-02-13T07:06:44.756 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 d3569347-f7c6-4f21-8726-e1cc18a8f406 2026-02-13T07:06:44.757 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d3569347-f7c6-4f21-8726-e1cc18a8f406' 2026-02-13T07:06:44.757 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:06:44.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCEzY5pF/R2LRAAPUrARuqD/5Ke4Tqo65bEDA== 2026-02-13T07:06:44.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCEzY5pF/R2LRAAPUrARuqD/5Ke4Tqo65bEDA=="}' 2026-02-13T07:06:44.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d3569347-f7c6-4f21-8726-e1cc18a8f406 -i td/ver-health/2/new.json 2026-02-13T07:06:44.869 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:06:44.875 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ver-health/2/new.json 2026-02-13T07:06:44.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/2 --osd-journal=td/ver-health/2/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCEzY5pF/R2LRAAPUrARuqD/5Ke4Tqo65bEDA== --osd-uuid d3569347-f7c6-4f21-8726-e1cc18a8f406 2026-02-13T07:06:44.893 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:44.890+0000 7f957c1c1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:44.894 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:44.892+0000 7f957c1c1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:44.895 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:44.893+0000 7f957c1c1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:44.896 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:44.893+0000 7f957c1c1900 -1 bdev(0x55585991ec00 td/ver-health/2/block) open stat got: (1) Operation not permitted 2026-02-13T07:06:44.896 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:44.893+0000 7f957c1c1900 -1 bluestore(td/ver-health/2) _read_fsid unparsable uuid 2026-02-13T07:06:45.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ver-health/2/keyring 2026-02-13T07:06:45.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:06:45.242 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T07:06:45.243 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T07:06:45.243 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ver-health/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:06:45.348 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:06:45.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T07:06:45.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/2 --osd-journal=td/ver-health/2/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:06:45.349 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:06:45.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:06:45.350 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:06:45.365 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:45.362+0000 7f3a58c27900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:45.366 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:45.363+0000 7f3a58c27900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:45.367 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:45.364+0000 7f3a58c27900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:45.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T07:06:45.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:06:45.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:06:45.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:06:45.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:06:45.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:45.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:06:45.448 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:06:45.448 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:45.448 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:06:45.546 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:06:45.560 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:45.557+0000 7f3a58c27900 -1 Falling back to public interface 2026-02-13T07:06:45.802 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:45.799+0000 7f3a58c27900 -1 osd.2 0 log_to_monitors true 2026-02-13T07:06:46.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:06:46.547 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:46.547 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:06:46.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:06:46.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:46.548 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:06:46.645 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/4022656160,v1:127.0.0.1:6817/4022656160] [v2:127.0.0.1:6818/4022656160,v1:127.0.0.1:6819/4022656160] exists,up d3569347-f7c6-4f21-8726-e1cc18a8f406 2026-02-13T07:06:46.645 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:06:46.645 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:06:46.645 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:06:46.645 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:82: TEST_check_version_health_1: sleep 5 2026-02-13T07:06:51.646 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:83: TEST_check_version_health_1: ceph health detail 2026-02-13T07:06:51.743 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_OK 2026-02-13T07:06:51.751 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:85: TEST_check_version_health_1: ceph health detail 2026-02-13T07:06:51.751 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:85: TEST_check_version_health_1: grep DAEMON_OLD_VERSION 2026-02-13T07:06:51.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:87: TEST_check_version_health_1: kill_daemons td/ver-health KILL osd.1 2026-02-13T07:06:51.854 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:06:51.855 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:06:51.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:06:51.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:06:51.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:06:51.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:06:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:88: TEST_check_version_health_1: ceph_debug_version_for_testing=01.00.00-gversion-test 2026-02-13T07:06:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:88: TEST_check_version_health_1: activate_osd td/ver-health 1 2026-02-13T07:06:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ver-health 2026-02-13T07:06:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:06:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T07:06:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:06:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ver-health/1 2026-02-13T07:06:51.964 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:06:51.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:06:51.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:06:51.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:06:51.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ver-health/1' 2026-02-13T07:06:51.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ver-health/1/journal' 2026-02-13T07:06:51.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:06:51.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:06:51.965 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:06:51.965 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:06:51.966 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:06:51.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:06:51.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:06:51.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:06:51.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:06:51.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:06:51.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:06:51.967 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ver-health/1 2026-02-13T07:06:51.968 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:06:51.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T07:06:51.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/1 --osd-journal=td/ver-health/1/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:06:51.969 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ver-health/1/whoami 2026-02-13T07:06:51.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T07:06:51.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:06:51.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:06:51.969 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:06:51.980 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:51.977+0000 7f2e45265900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:51.981 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:51.978+0000 7f2e45265900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:51.982 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:51.979+0000 7f2e45265900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:52.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T07:06:52.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:06:52.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:06:52.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:06:52.066 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:06:52.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:52.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:06:52.067 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:06:52.068 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:52.068 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:06:52.157 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:52.155+0000 7f2e45265900 -1 Falling back to public interface 2026-02-13T07:06:52.162 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:06:52.437 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:52.434+0000 7f2e45265900 -1 osd.1 14 log_to_monitors true 2026-02-13T07:06:53.163 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:06:53.164 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:53.164 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:06:53.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:06:53.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:53.165 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:06:53.264 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 17 up_thru 0 down_at 15 last_clean_interval [9,14) [v2:127.0.0.1:6808/1440627555,v1:127.0.0.1:6809/1440627555] [v2:127.0.0.1:6810/1440627555,v1:127.0.0.1:6811/1440627555] exists,up a235b734-ab7c-451b-8dfd-ed0349ad778e 2026-02-13T07:06:53.265 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:90: TEST_check_version_health_1: wait_for_health_string 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:40: wait_for_health_string: local 'grep_string=HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:41: wait_for_health_string: local seconds=20 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:44: wait_for_health_string: set -o pipefail 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:45: wait_for_health_string: PASSED=false 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i=0 )) 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:06:53.266 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:06:53.363 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:06:54.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:06:54.364 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:06:54.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:06:54.365 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:06:54.463 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:06:55.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:06:55.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:06:55.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:06:55.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:06:55.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:49: wait_for_health_string: PASSED=true 2026-02-13T07:06:55.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:50: wait_for_health_string: break 2026-02-13T07:06:55.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:54: wait_for_health_string: set +o pipefail 2026-02-13T07:06:55.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:57: wait_for_health_string: '[' true = false ']' 2026-02-13T07:06:55.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:61: wait_for_health_string: return 0 2026-02-13T07:06:55.564 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:92: TEST_check_version_health_1: ceph health detail 2026-02-13T07:06:55.658 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_WARN There is a daemon running an older version of ceph 2026-02-13T07:06:55.658 INFO:tasks.workunit.client.0.trial040.stdout:[WRN] DAEMON_OLD_VERSION: There is a daemon running an older version of ceph 2026-02-13T07:06:55.658 INFO:tasks.workunit.client.0.trial040.stdout: osd.1 is running an older version of ceph: 01.00.00-gversion-test 2026-02-13T07:06:55.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:94: TEST_check_version_health_1: ceph health detail 2026-02-13T07:06:55.664 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:94: TEST_check_version_health_1: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:06:55.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:95: TEST_check_version_health_1: ceph health detail 2026-02-13T07:06:55.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:95: TEST_check_version_health_1: grep -q '^[[]WRN[]] DAEMON_OLD_VERSION: There is a daemon running an older version of ceph' 2026-02-13T07:06:55.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:96: TEST_check_version_health_1: ceph health detail 2026-02-13T07:06:55.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:96: TEST_check_version_health_1: grep -q 'osd.1 is running an older version of ceph: 01.00.00-gversion-test' 2026-02-13T07:06:55.961 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:98: TEST_check_version_health_1: kill_daemons td/ver-health KILL osd.2 2026-02-13T07:06:55.962 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:06:55.962 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:06:55.962 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:06:55.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:06:55.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:06:56.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:06:56.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:99: TEST_check_version_health_1: ceph_debug_version_for_testing=01.00.00-gversion-test 2026-02-13T07:06:56.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:99: TEST_check_version_health_1: activate_osd td/ver-health 2 2026-02-13T07:06:56.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ver-health 2026-02-13T07:06:56.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:06:56.071 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T07:06:56.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:06:56.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ver-health/2 2026-02-13T07:06:56.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:06:56.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:06:56.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:06:56.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:06:56.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ver-health/2' 2026-02-13T07:06:56.072 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ver-health/2/journal' 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:56.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ver-health/2 2026-02-13T07:06:56.074 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T07:06:56.075 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:06:56.076 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/2 --osd-journal=td/ver-health/2/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:06:56.076 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ver-health/2/whoami 2026-02-13T07:06:56.076 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T07:06:56.076 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:06:56.076 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:06:56.076 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:06:56.086 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:56.083+0000 7f8bf5afa900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:56.087 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:56.085+0000 7f8bf5afa900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:56.088 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:56.086+0000 7f8bf5afa900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:56.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T07:06:56.170 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:06:56.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:06:56.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:06:56.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:06:56.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:56.171 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:06:56.171 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:06:56.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:56.172 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:06:56.261 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:56.259+0000 7f8bf5afa900 -1 Falling back to public interface 2026-02-13T07:06:56.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:06:56.562 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:56.558+0000 7f8bf5afa900 -1 osd.2 17 log_to_monitors true 2026-02-13T07:06:57.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:06:57.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:57.269 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:06:57.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:06:57.269 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:57.270 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:06:57.370 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 20 up_thru 0 down_at 18 last_clean_interval [14,17) [v2:127.0.0.1:6816/2915416514,v1:127.0.0.1:6817/2915416514] [v2:127.0.0.1:6818/2915416514,v1:127.0.0.1:6819/2915416514] exists,up d3569347-f7c6-4f21-8726-e1cc18a8f406 2026-02-13T07:06:57.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:06:57.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:06:57.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:06:57.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:100: TEST_check_version_health_1: kill_daemons td/ver-health KILL osd.0 2026-02-13T07:06:57.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:06:57.370 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:06:57.370 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:06:57.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:06:57.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:101: TEST_check_version_health_1: ceph_debug_version_for_testing=02.00.00-gversion-test 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:101: TEST_check_version_health_1: activate_osd td/ver-health 0 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ver-health 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ver-health/0 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ver-health/0' 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ver-health/0/journal' 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:06:57.477 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:06:57.478 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:06:57.478 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:06:57.478 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:06:57.478 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:06:57.478 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:06:57.478 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:06:57.480 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:57.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:06:57.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:06:57.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:06:57.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:06:57.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:06:57.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:06:57.480 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:06:57.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:06:57.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:06:57.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:06:57.481 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ver-health/0 2026-02-13T07:06:57.481 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:06:57.482 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T07:06:57.482 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/0 --osd-journal=td/ver-health/0/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:06:57.482 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ver-health/0/whoami 2026-02-13T07:06:57.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T07:06:57.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:06:57.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:06:57.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:06:57.497 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:57.494+0000 7f04d43dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:57.498 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:57.495+0000 7f04d43dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:57.499 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:57.496+0000 7f04d43dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:06:57.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T07:06:57.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:06:57.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:06:57.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:06:57.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:06:57.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:57.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:06:57.583 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:06:57.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:57.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:06:57.671 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:57.668+0000 7f04d43dc900 -1 Falling back to public interface 2026-02-13T07:06:57.678 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:06:57.963 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:06:57.960+0000 7f04d43dc900 -1 osd.0 20 log_to_monitors true 2026-02-13T07:06:58.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:06:58.680 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:06:58.680 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:06:58.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:06:58.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:06:58.681 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:06:58.778 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 24 up_thru 0 down_at 21 last_clean_interval [5,20) [v2:127.0.0.1:6800/622796130,v1:127.0.0.1:6801/622796130] [v2:127.0.0.1:6802/622796130,v1:127.0.0.1:6803/622796130] exists,up 01283a02-6bdb-4e4b-8e2b-135b45da8566 2026-02-13T07:06:58.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:06:58.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:06:58.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:06:58.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:103: TEST_check_version_health_1: wait_for_health_string 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:06:58.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:40: wait_for_health_string: local 'grep_string=HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:06:58.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:41: wait_for_health_string: local seconds=20 2026-02-13T07:06:58.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:44: wait_for_health_string: set -o pipefail 2026-02-13T07:06:58.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:45: wait_for_health_string: PASSED=false 2026-02-13T07:06:58.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i=0 )) 2026-02-13T07:06:58.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:06:58.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:06:58.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:06:58.875 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:06:59.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:06:59.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:06:59.876 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:06:59.877 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:06:59.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:49: wait_for_health_string: PASSED=true 2026-02-13T07:06:59.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:50: wait_for_health_string: break 2026-02-13T07:06:59.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:54: wait_for_health_string: set +o pipefail 2026-02-13T07:06:59.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:57: wait_for_health_string: '[' true = false ']' 2026-02-13T07:06:59.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:61: wait_for_health_string: return 0 2026-02-13T07:06:59.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:105: TEST_check_version_health_1: ceph health detail 2026-02-13T07:07:00.073 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_ERR There are daemons running multiple old versions of ceph 2026-02-13T07:07:00.073 INFO:tasks.workunit.client.0.trial040.stdout:[ERR] DAEMON_OLD_VERSION: There are daemons running multiple old versions of ceph 2026-02-13T07:07:00.073 INFO:tasks.workunit.client.0.trial040.stdout: osd.1 osd.2 are running an older version of ceph: 01.00.00-gversion-test 2026-02-13T07:07:00.073 INFO:tasks.workunit.client.0.trial040.stdout: osd.0 is running an older version of ceph: 02.00.00-gversion-test 2026-02-13T07:07:00.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:106: TEST_check_version_health_1: ceph health detail 2026-02-13T07:07:00.079 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:106: TEST_check_version_health_1: grep -q 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:07:00.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:107: TEST_check_version_health_1: ceph health detail 2026-02-13T07:07:00.178 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:107: TEST_check_version_health_1: grep -q '^[[]ERR[]] DAEMON_OLD_VERSION: There are daemons running multiple old versions of ceph' 2026-02-13T07:07:00.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:108: TEST_check_version_health_1: ceph health detail 2026-02-13T07:07:00.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:108: TEST_check_version_health_1: grep -q 'osd.1 osd.2 are running an older version of ceph: 01.00.00-gversion-test' 2026-02-13T07:07:00.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:109: TEST_check_version_health_1: ceph health detail 2026-02-13T07:07:00.382 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:109: TEST_check_version_health_1: grep -q 'osd.0 is running an older version of ceph: 02.00.00-gversion-test' 2026-02-13T07:07:00.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:35: run: teardown td/ver-health 2026-02-13T07:07:00.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:07:00.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:07:00.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:07:00.483 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:00.483 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:00.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:00.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:00.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:00.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:00.590 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:07:00.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:07:00.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:07:00.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:07:00.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:07:00.593 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:07:00.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:00.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:07:00.594 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:07:00.594 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:00.595 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:07:00.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:07:00.596 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:07:00.605 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:07:00.605 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.605 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.605 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:07:00.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:07:00.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:07:00.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:32: run: for func in $funcs 2026-02-13T07:07:00.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:33: run: setup td/ver-health 2026-02-13T07:07:00.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ver-health 2026-02-13T07:07:00.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ver-health 2026-02-13T07:07:00.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:07:00.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:07:00.606 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:07:00.607 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:00.607 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:00.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:00.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:00.607 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:00.608 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:00.608 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:07:00.609 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:07:00.609 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:07:00.610 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:07:00.610 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:07:00.610 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:07:00.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:00.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:07:00.611 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:07:00.612 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:00.612 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:07:00.613 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:07:00.613 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:07:00.614 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:07:00.614 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.614 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:07:00.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:07:00.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:07:00.615 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ver-health 2026-02-13T07:07:00.616 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:07:00.617 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.617 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.617 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.164153 2026-02-13T07:07:00.618 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ver-health 1' TERM HUP INT 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:34: run: TEST_check_version_health_2 td/ver-health 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:114: TEST_check_version_health_2: local dir=td/ver-health 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:117: TEST_check_version_health_2: CEPH_ARGS='--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:119: TEST_check_version_health_2: setup td/ver-health 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ver-health 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ver-health 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:00.619 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:00.620 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:00.620 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:07:00.621 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:07:00.621 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:07:00.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:07:00.622 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:07:00.622 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:07:00.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:00.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:07:00.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:07:00.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:00.624 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:07:00.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:07:00.625 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:07:00.626 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:07:00.626 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.626 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:07:00.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:07:00.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:07:00.627 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ver-health 2026-02-13T07:07:00.628 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:07:00.628 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.629 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.629 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.164153 2026-02-13T07:07:00.630 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:07:00.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:07:00.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:07:00.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ver-health 1' TERM HUP INT 2026-02-13T07:07:00.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:122: TEST_check_version_health_2: run_mon td/ver-health a --public-addr=127.0.0.1:7165 --mon_warn_older_version_delay=0 2026-02-13T07:07:00.630 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ver-health 2026-02-13T07:07:00.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:07:00.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:07:00.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:07:00.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ver-health/a 2026-02-13T07:07:00.631 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ver-health/a --run-dir=td/ver-health --public-addr=127.0.0.1:7165 --mon_warn_older_version_delay=0 2026-02-13T07:07:00.652 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:07:00.652 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:00.652 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:00.653 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:00.653 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.653 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.653 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:00.653 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ver-health/a '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --mon-cluster-log-file=td/ver-health/log --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7165 --mon_warn_older_version_delay=0 2026-02-13T07:07:00.679 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:07:00.680 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:07:00.680 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:00.680 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:07:00.680 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:07:00.680 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:07:00.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:07:00.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:07:00.680 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:07:00.681 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:00.681 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.682 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.682 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.a.asok 2026-02-13T07:07:00.682 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:00.682 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.a.asok config get fsid 2026-02-13T07:07:00.727 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:07:00.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:00.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:07:00.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:07:00.728 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:07:00.729 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:07:00.729 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:07:00.729 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:07:00.729 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:00.729 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.729 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.730 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.a.asok 2026-02-13T07:07:00.730 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:00.730 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.a.asok config get mon_host 2026-02-13T07:07:00.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:123: TEST_check_version_health_2: run_mon td/ver-health b --public-addr=127.0.0.1:7166 --mon_warn_older_version_delay=0 2026-02-13T07:07:00.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ver-health 2026-02-13T07:07:00.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:07:00.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T07:07:00.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:07:00.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ver-health/b 2026-02-13T07:07:00.773 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ver-health/b --run-dir=td/ver-health --public-addr=127.0.0.1:7166 --mon_warn_older_version_delay=0 2026-02-13T07:07:00.792 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:07:00.793 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:00.793 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:00.793 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:00.793 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.793 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.793 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:00.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ver-health/b '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --mon-cluster-log-file=td/ver-health/log --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7166 --mon_warn_older_version_delay=0 2026-02-13T07:07:00.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:07:00.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T07:07:00.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:00.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T07:07:00.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:07:00.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:07:00.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T07:07:00.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T07:07:00.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T07:07:00.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:00.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.822 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.822 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.b.asok 2026-02-13T07:07:00.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:00.822 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.b.asok config get fsid 2026-02-13T07:07:00.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T07:07:00.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:00.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T07:07:00.867 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:07:00.868 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:07:00.868 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T07:07:00.868 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T07:07:00.868 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T07:07:00.868 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:00.868 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.868 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.868 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.b.asok 2026-02-13T07:07:00.869 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:00.869 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.b.asok config get mon_host 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:124: TEST_check_version_health_2: run_osd td/ver-health 0 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ver-health 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ver-health/0 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:07:00.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ver-health/0' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ver-health/0/journal' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:07:00.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ver-health/0 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:07:00.917 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f65f355d-568d-433a-be1d-a79d606190de 2026-02-13T07:07:00.918 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 f65f355d-568d-433a-be1d-a79d606190de 2026-02-13T07:07:00.918 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f65f355d-568d-433a-be1d-a79d606190de' 2026-02-13T07:07:00.918 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:07:00.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCUzY5pxYIcNxAAaqoAQ+rlPhNvcHV8rJ+IvA== 2026-02-13T07:07:00.926 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCUzY5pxYIcNxAAaqoAQ+rlPhNvcHV8rJ+IvA=="}' 2026-02-13T07:07:00.927 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f65f355d-568d-433a-be1d-a79d606190de -i td/ver-health/0/new.json 2026-02-13T07:07:07.030 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:07.037 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ver-health/0/new.json 2026-02-13T07:07:07.038 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/0 --osd-journal=td/ver-health/0/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCUzY5pxYIcNxAAaqoAQ+rlPhNvcHV8rJ+IvA== --osd-uuid f65f355d-568d-433a-be1d-a79d606190de 2026-02-13T07:07:07.056 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.053+0000 7f90d140e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:07.057 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.055+0000 7f90d140e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:07.057 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.056+0000 7f90d140e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:07.058 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.056+0000 7f90d140e900 -1 bdev(0x563272236c00 td/ver-health/0/block) open stat got: (1) Operation not permitted 2026-02-13T07:07:07.058 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.056+0000 7f90d140e900 -1 bluestore(td/ver-health/0) _read_fsid unparsable uuid 2026-02-13T07:07:07.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ver-health/0/keyring 2026-02-13T07:07:07.387 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:07:07.388 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:07:07.388 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:07:07.389 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ver-health/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:07:07.491 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:07:07.491 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:07:07.492 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/0 --osd-journal=td/ver-health/0/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:07:07.492 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:07:07.492 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:07:07.492 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:07:07.507 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.505+0000 7f1dd11c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:07.508 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.506+0000 7f1dd11c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:07.509 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.507+0000 7f1dd11c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:07.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:07:07.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:07:07.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:07:07.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:07:07.589 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:07:07.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:07.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:07:07.590 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:07.590 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:07.591 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:07:07.662 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.660+0000 7f1dd11c6900 -1 Falling back to public interface 2026-02-13T07:07:07.688 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:07:07.904 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:07.902+0000 7f1dd11c6900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:07:08.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:07:08.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:08.690 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:08.691 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:07:08.691 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:08.691 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:07:08.789 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/237610129,v1:127.0.0.1:6801/237610129] [v2:127.0.0.1:6802/237610129,v1:127.0.0.1:6803/237610129] exists,up f65f355d-568d-433a-be1d-a79d606190de 2026-02-13T07:07:08.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:07:08.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:07:08.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:07:08.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:125: TEST_check_version_health_2: run_osd td/ver-health 1 2026-02-13T07:07:08.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ver-health 2026-02-13T07:07:08.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:07:08.792 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ver-health/1 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ver-health/1' 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ver-health/1/journal' 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:07:08.793 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:08.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:07:08.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:07:08.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:07:08.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:07:08.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:07:08.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:07:08.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ver-health/1 2026-02-13T07:07:08.795 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:07:08.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0fef9d11-2699-4222-bed1-1aed337245b8 2026-02-13T07:07:08.796 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 0fef9d11-2699-4222-bed1-1aed337245b8 2026-02-13T07:07:08.796 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 0fef9d11-2699-4222-bed1-1aed337245b8' 2026-02-13T07:07:08.796 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:07:08.804 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCczY5pf77XLxAA2VVKP5TYaJKOcca/ifQ7dw== 2026-02-13T07:07:08.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCczY5pf77XLxAA2VVKP5TYaJKOcca/ifQ7dw=="}' 2026-02-13T07:07:08.805 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0fef9d11-2699-4222-bed1-1aed337245b8 -i td/ver-health/1/new.json 2026-02-13T07:07:08.909 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:08.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ver-health/1/new.json 2026-02-13T07:07:08.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/1 --osd-journal=td/ver-health/1/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCczY5pf77XLxAA2VVKP5TYaJKOcca/ifQ7dw== --osd-uuid 0fef9d11-2699-4222-bed1-1aed337245b8 2026-02-13T07:07:08.934 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:08.933+0000 7fd81f20d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:08.936 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:08.934+0000 7fd81f20d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:08.937 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:08.935+0000 7fd81f20d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:08.937 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:08.936+0000 7fd81f20d900 -1 bdev(0x556bee18cc00 td/ver-health/1/block) open stat got: (1) Operation not permitted 2026-02-13T07:07:08.938 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:08.936+0000 7fd81f20d900 -1 bluestore(td/ver-health/1) _read_fsid unparsable uuid 2026-02-13T07:07:09.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ver-health/1/keyring 2026-02-13T07:07:09.267 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:07:09.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:07:09.268 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:07:09.268 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ver-health/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:07:09.370 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:07:09.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:07:09.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/1 --osd-journal=td/ver-health/1/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:07:09.371 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:07:09.372 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:07:09.372 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:07:09.388 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:09.386+0000 7f170780e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:09.389 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:09.388+0000 7f170780e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:09.390 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:09.389+0000 7f170780e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:09.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:07:09.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:07:09.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:07:09.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:07:09.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:07:09.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:09.469 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:09.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:07:09.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:09.470 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:07:09.539 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:09.537+0000 7f170780e900 -1 Falling back to public interface 2026-02-13T07:07:09.567 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:07:09.810 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:09.808+0000 7f170780e900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:07:10.568 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:07:10.568 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:10.569 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:10.569 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:07:10.570 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:10.570 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:07:10.669 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/3268060064,v1:127.0.0.1:6809/3268060064] [v2:127.0.0.1:6810/3268060064,v1:127.0.0.1:6811/3268060064] exists,up 0fef9d11-2699-4222-bed1-1aed337245b8 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:126: TEST_check_version_health_2: run_osd td/ver-health 2 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ver-health 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ver-health/2 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ver-health/2' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ver-health/2/journal' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:10.673 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ver-health/2 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b65553d4-b786-49ff-82e8-dadf440aa4e1 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b65553d4-b786-49ff-82e8-dadf440aa4e1' 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 b65553d4-b786-49ff-82e8-dadf440aa4e1 2026-02-13T07:07:10.674 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:07:10.684 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCezY5pFiCuKBAAze1stGE6QdvtMlGHCJ0AgA== 2026-02-13T07:07:10.684 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCezY5pFiCuKBAAze1stGE6QdvtMlGHCJ0AgA=="}' 2026-02-13T07:07:10.684 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b65553d4-b786-49ff-82e8-dadf440aa4e1 -i td/ver-health/2/new.json 2026-02-13T07:07:10.787 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:07:10.794 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ver-health/2/new.json 2026-02-13T07:07:10.795 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/2 --osd-journal=td/ver-health/2/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCezY5pFiCuKBAAze1stGE6QdvtMlGHCJ0AgA== --osd-uuid b65553d4-b786-49ff-82e8-dadf440aa4e1 2026-02-13T07:07:10.813 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:10.811+0000 7f708251c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:10.814 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:10.813+0000 7f708251c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:10.815 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:10.814+0000 7f708251c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:10.816 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:10.814+0000 7f708251c900 -1 bdev(0x564c72f66c00 td/ver-health/2/block) open stat got: (1) Operation not permitted 2026-02-13T07:07:10.816 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:10.814+0000 7f708251c900 -1 bluestore(td/ver-health/2) _read_fsid unparsable uuid 2026-02-13T07:07:11.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ver-health/2/keyring 2026-02-13T07:07:11.151 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:07:11.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T07:07:11.152 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T07:07:11.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ver-health/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:07:11.256 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:07:11.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T07:07:11.256 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/2 --osd-journal=td/ver-health/2/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:07:11.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:07:11.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:07:11.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:07:11.273 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:11.272+0000 7f8bd9f8a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:11.275 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:11.273+0000 7f8bd9f8a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:11.276 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:11.274+0000 7f8bd9f8a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:11.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T07:07:11.358 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:07:11.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:07:11.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:07:11.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:07:11.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:11.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:07:11.359 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:11.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:11.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:07:11.455 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:11.454+0000 7f8bd9f8a900 -1 Falling back to public interface 2026-02-13T07:07:11.456 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:07:11.697 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:11.695+0000 7f8bd9f8a900 -1 osd.2 0 log_to_monitors true 2026-02-13T07:07:12.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:07:12.457 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:12.458 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:12.458 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:07:12.459 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:12.459 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:07:12.556 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/1426954980,v1:127.0.0.1:6817/1426954980] [v2:127.0.0.1:6818/1426954980,v1:127.0.0.1:6819/1426954980] exists,up b65553d4-b786-49ff-82e8-dadf440aa4e1 2026-02-13T07:07:12.557 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:07:12.557 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:07:12.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:07:12.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:127: TEST_check_version_health_2: run_mgr td/ver-health x 2026-02-13T07:07:12.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ver-health 2026-02-13T07:07:12.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:07:12.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T07:07:12.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:07:12.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ver-health/x 2026-02-13T07:07:12.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:07:12.657 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:07:12.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:12.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:12.658 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:12.658 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:12.658 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:12.658 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:12.659 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:07:12.659 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ver-health/x '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:07:12.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:128: TEST_check_version_health_2: run_mgr td/ver-health y 2026-02-13T07:07:12.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ver-health 2026-02-13T07:07:12.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T07:07:12.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=y 2026-02-13T07:07:12.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T07:07:12.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ver-health/y 2026-02-13T07:07:12.676 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T07:07:12.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T07:07:12.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:12.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:12.778 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:12.778 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:12.778 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:12.778 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:12.779 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:07:12.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id y --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ver-health/y '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T07:07:12.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:129: TEST_check_version_health_2: run_mds td/ver-health m 2026-02-13T07:07:12.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ver-health 2026-02-13T07:07:12.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-13T07:07:12.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=m 2026-02-13T07:07:12.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-13T07:07:12.798 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ver-health/m 2026-02-13T07:07:12.799 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-13T07:07:12.799 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:12.799 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:12.799 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:12.799 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:12.799 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:12.799 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:12.799 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id m --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ver-health/m '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:12.816 INFO:tasks.workunit.client.0.trial040.stdout:starting mds.m at 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:130: TEST_check_version_health_2: run_mds td/ver-health n 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ver-health 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=n 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ver-health/n 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:12.819 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id n --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ver-health/n '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:12.835 INFO:tasks.workunit.client.0.trial040.stdout:starting mds.n at 2026-02-13T07:07:12.838 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:132: TEST_check_version_health_2: sleep 5 2026-02-13T07:07:17.840 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:133: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:18.145 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_OK 2026-02-13T07:07:18.152 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:135: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:18.153 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:135: TEST_check_version_health_2: grep DAEMON_OLD_VERSION 2026-02-13T07:07:18.467 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:137: TEST_check_version_health_2: kill_daemons td/ver-health KILL mon.b 2026-02-13T07:07:18.468 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:18.468 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:18.468 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:18.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:18.469 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:18.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:18.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:138: TEST_check_version_health_2: ceph_debug_version_for_testing=01.00.00-gversion-test 2026-02-13T07:07:18.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:138: TEST_check_version_health_2: run_mon td/ver-health b --mon_warn_older_version_delay=0 2026-02-13T07:07:18.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ver-health 2026-02-13T07:07:18.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:07:18.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T07:07:18.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:07:18.575 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ver-health/b 2026-02-13T07:07:18.576 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ver-health/b --run-dir=td/ver-health --mon_warn_older_version_delay=0 2026-02-13T07:07:18.589 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:18.587+0000 7f73b2218d80 -1 'td/ver-health/b' already exists and is not empty: monitor may already exist 2026-02-13T07:07:18.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:07:18.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:18.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:18.591 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:18.591 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:18.591 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:18.591 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:18.592 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ver-health/b '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --mon-cluster-log-file=td/ver-health/log --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --mon_warn_older_version_delay=0 2026-02-13T07:07:18.623 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:07:18.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T07:07:18.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:18.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T07:07:18.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:07:18.624 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:07:18.624 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T07:07:18.625 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T07:07:18.625 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T07:07:18.625 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:18.625 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:18.625 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:18.625 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.b.asok 2026-02-13T07:07:18.625 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:18.625 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.b.asok config get fsid 2026-02-13T07:07:18.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T07:07:18.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:18.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T07:07:18.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:07:18.669 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:07:18.670 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T07:07:18.670 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T07:07:18.670 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T07:07:18.670 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:18.670 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:18.670 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:18.670 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.b.asok 2026-02-13T07:07:18.671 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:18.671 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.b.asok config get mon_host 2026-02-13T07:07:18.714 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:142: TEST_check_version_health_2: kill_daemons td/ver-health KILL mds.m 2026-02-13T07:07:18.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:18.715 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:18.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:18.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:18.715 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:18.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:18.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:143: TEST_check_version_health_2: ceph_debug_version_for_testing=01.00.00-gversion-test 2026-02-13T07:07:18.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:143: TEST_check_version_health_2: run_mds td/ver-health m 2026-02-13T07:07:18.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ver-health 2026-02-13T07:07:18.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-13T07:07:18.820 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=m 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ver-health/m 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:18.821 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id m --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ver-health/m '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:18.838 INFO:tasks.workunit.client.0.trial040.stdout:starting mds.m at 2026-02-13T07:07:18.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:145: TEST_check_version_health_2: wait_for_health_string 'HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:18.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:40: wait_for_health_string: local 'grep_string=HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:18.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:41: wait_for_health_string: local seconds=20 2026-02-13T07:07:18.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:44: wait_for_health_string: set -o pipefail 2026-02-13T07:07:18.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:45: wait_for_health_string: PASSED=false 2026-02-13T07:07:18.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i=0 )) 2026-02-13T07:07:18.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:07:18.841 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:07:18.842 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:19.132 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:07:20.133 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:07:20.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:07:20.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:07:20.134 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:20.419 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:07:21.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:07:21.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:07:21.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:07:21.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:21.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:07:22.707 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:07:22.707 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:07:22.707 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:07:22.707 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:22.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:07:23.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:07:23.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:07:23.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:07:23.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:24.279 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:07:25.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:07:25.280 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:07:25.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:07:25.281 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:25.565 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:07:26.567 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:07:26.567 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:07:26.567 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:07:26.568 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:26.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:49: wait_for_health_string: PASSED=true 2026-02-13T07:07:26.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:50: wait_for_health_string: break 2026-02-13T07:07:26.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:54: wait_for_health_string: set +o pipefail 2026-02-13T07:07:26.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:57: wait_for_health_string: '[' true = false ']' 2026-02-13T07:07:26.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:61: wait_for_health_string: return 0 2026-02-13T07:07:26.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:147: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:27.168 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_WARN There are daemons running an older version of ceph 2026-02-13T07:07:27.168 INFO:tasks.workunit.client.0.trial040.stdout:[WRN] DAEMON_OLD_VERSION: There are daemons running an older version of ceph 2026-02-13T07:07:27.169 INFO:tasks.workunit.client.0.trial040.stdout: mon.b mds.m are running an older version of ceph: 01.00.00-gversion-test 2026-02-13T07:07:27.176 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:149: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:27.177 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:149: TEST_check_version_health_2: grep -q 'HEALTH_WARN .*There are daemons running an older version of ceph' 2026-02-13T07:07:27.488 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:150: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:27.488 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:150: TEST_check_version_health_2: grep -q '^[[]WRN[]] DAEMON_OLD_VERSION: There are daemons running an older version of ceph' 2026-02-13T07:07:27.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:151: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:27.809 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:151: TEST_check_version_health_2: grep -q 'mon.b mds.m are running an older version of ceph: 01.00.00-gversion-test' 2026-02-13T07:07:28.127 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:153: TEST_check_version_health_2: kill_daemons td/ver-health KILL osd.2 2026-02-13T07:07:28.128 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:28.128 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:28.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:28.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:28.128 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:28.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:28.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:154: TEST_check_version_health_2: ceph_debug_version_for_testing=01.00.00-gversion-test 2026-02-13T07:07:28.236 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:154: TEST_check_version_health_2: activate_osd td/ver-health 2 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ver-health 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ver-health/2 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ver-health/2' 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ver-health/2/journal' 2026-02-13T07:07:28.237 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:07:28.238 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:07:28.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:28.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:07:28.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:07:28.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:07:28.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:07:28.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:07:28.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:07:28.239 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ver-health/2 2026-02-13T07:07:28.240 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:07:28.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T07:07:28.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/2 --osd-journal=td/ver-health/2/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:07:28.241 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ver-health/2/whoami 2026-02-13T07:07:28.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T07:07:28.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:07:28.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:07:28.241 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:07:28.254 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:28.252+0000 7fd2ad4ea900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:28.255 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:28.253+0000 7fd2ad4ea900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:28.256 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:28.254+0000 7fd2ad4ea900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:28.387 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:28.385+0000 7fd2ad4ea900 -1 Falling back to public interface 2026-02-13T07:07:28.463 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T07:07:28.463 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:07:28.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:07:28.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:07:28.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:07:28.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:28.464 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:07:28.464 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:28.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:28.465 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:07:28.689 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:07:28.736 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:28.735+0000 7fd2ad4ea900 -1 osd.2 17 log_to_monitors true 2026-02-13T07:07:29.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:07:29.690 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:29.691 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:07:29.691 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:29.691 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:29.692 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:07:29.914 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 20 up_thru 0 down_at 18 last_clean_interval [15,17) [v2:127.0.0.1:6816/728275838,v1:127.0.0.1:6817/728275838] [v2:127.0.0.1:6818/728275838,v1:127.0.0.1:6819/728275838] exists,up b65553d4-b786-49ff-82e8-dadf440aa4e1 2026-02-13T07:07:29.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:07:29.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:07:29.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:07:29.915 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:155: TEST_check_version_health_2: kill_daemons td/ver-health KILL osd.0 2026-02-13T07:07:29.915 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:29.916 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:29.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:29.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:29.916 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:30.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:30.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:156: TEST_check_version_health_2: ceph_debug_version_for_testing=02.00.00-gversion-test 2026-02-13T07:07:30.023 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:156: TEST_check_version_health_2: activate_osd td/ver-health 0 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ver-health 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ver-health/0 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ver-health/0' 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ver-health/0/journal' 2026-02-13T07:07:30.024 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:07:30.025 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:30.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:07:30.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:07:30.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:07:30.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:07:30.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:07:30.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:07:30.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ver-health/0 2026-02-13T07:07:30.026 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T07:07:30.026 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:07:30.027 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/0 --osd-journal=td/ver-health/0/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:07:30.027 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ver-health/0/whoami 2026-02-13T07:07:30.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T07:07:30.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:07:30.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:07:30.028 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:07:30.038 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:30.037+0000 7f48ab093900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:30.040 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:30.038+0000 7f48ab093900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:30.040 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:30.039+0000 7f48ab093900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:30.186 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:30.185+0000 7f48ab093900 -1 Falling back to public interface 2026-02-13T07:07:30.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T07:07:30.250 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:07:30.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:07:30.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:07:30.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:07:30.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:30.251 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:07:30.251 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:30.252 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:30.252 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:07:30.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:07:30.508 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:30.507+0000 7f48ab093900 -1 osd.0 20 log_to_monitors true 2026-02-13T07:07:31.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:07:31.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:31.475 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:07:31.476 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:31.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:31.476 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:07:31.702 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 21 last_clean_interval [5,20) [v2:127.0.0.1:6800/1681755197,v1:127.0.0.1:6801/1681755197] [v2:127.0.0.1:6802/1681755197,v1:127.0.0.1:6803/1681755197] exists,up f65f355d-568d-433a-be1d-a79d606190de 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:158: TEST_check_version_health_2: wait_for_health_string 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:40: wait_for_health_string: local 'grep_string=HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:41: wait_for_health_string: local seconds=20 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:44: wait_for_health_string: set -o pipefail 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:45: wait_for_health_string: PASSED=false 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i=0 )) 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:07:31.704 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:07:31.705 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:07:31.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:49: wait_for_health_string: PASSED=true 2026-02-13T07:07:31.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:50: wait_for_health_string: break 2026-02-13T07:07:31.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:54: wait_for_health_string: set +o pipefail 2026-02-13T07:07:31.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:57: wait_for_health_string: '[' true = false ']' 2026-02-13T07:07:31.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:61: wait_for_health_string: return 0 2026-02-13T07:07:31.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:160: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:32.303 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_ERR There are daemons running multiple old versions of ceph 2026-02-13T07:07:32.303 INFO:tasks.workunit.client.0.trial040.stdout:[ERR] DAEMON_OLD_VERSION: There are daemons running multiple old versions of ceph 2026-02-13T07:07:32.303 INFO:tasks.workunit.client.0.trial040.stdout: mon.b osd.2 mds.m are running an older version of ceph: 01.00.00-gversion-test 2026-02-13T07:07:32.303 INFO:tasks.workunit.client.0.trial040.stdout: osd.0 is running an older version of ceph: 02.00.00-gversion-test 2026-02-13T07:07:32.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:161: TEST_check_version_health_2: ceph health 2026-02-13T07:07:32.311 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:161: TEST_check_version_health_2: grep -q 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:07:32.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:162: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:32.599 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:162: TEST_check_version_health_2: grep -q 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:07:32.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:163: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:32.914 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:163: TEST_check_version_health_2: grep -q '^[[]ERR[]] DAEMON_OLD_VERSION: There are daemons running multiple old versions of ceph' 2026-02-13T07:07:33.230 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:164: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:33.231 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:164: TEST_check_version_health_2: grep -q 'mon.b osd.2 mds.m are running an older version of ceph: 01.00.00-gversion-test' 2026-02-13T07:07:33.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:165: TEST_check_version_health_2: ceph health detail 2026-02-13T07:07:33.550 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:165: TEST_check_version_health_2: grep -q 'osd.0 is running an older version of ceph: 02.00.00-gversion-test' 2026-02-13T07:07:33.862 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:35: run: teardown td/ver-health 2026-02-13T07:07:33.862 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:07:33.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:07:33.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:07:33.863 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:33.863 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:33.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:33.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:33.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:33.972 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:33.972 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:07:33.973 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:07:33.974 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:07:33.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:07:33.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:07:33.976 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:07:33.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:33.976 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:07:33.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:07:33.977 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:33.977 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:07:33.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:07:33.978 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:07:33.987 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:07:33.987 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:33.987 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:33.987 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:07:33.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:07:33.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:07:33.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:32: run: for func in $funcs 2026-02-13T07:07:33.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:33: run: setup td/ver-health 2026-02-13T07:07:33.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ver-health 2026-02-13T07:07:33.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ver-health 2026-02-13T07:07:33.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:07:33.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:07:33.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:07:33.990 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:33.990 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:33.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:33.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:33.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:33.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:33.991 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:07:33.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:07:33.992 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:07:33.992 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:07:33.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:07:33.993 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:07:33.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:33.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:07:33.994 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:07:33.994 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:33.995 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:07:33.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:07:33.996 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:07:33.997 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:07:33.997 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:33.997 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:33.997 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:07:33.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:07:33.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:07:33.998 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ver-health 2026-02-13T07:07:33.999 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:07:33.999 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:33.999 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.000 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.164153 2026-02-13T07:07:34.000 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:07:34.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:07:34.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:07:34.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ver-health 1' TERM HUP INT 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:34: run: TEST_check_version_health_3 td/ver-health 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:170: TEST_check_version_health_3: local dir=td/ver-health 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:173: TEST_check_version_health_3: CEPH_ARGS='--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:175: TEST_check_version_health_3: setup td/ver-health 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ver-health 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ver-health 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:34.003 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:34.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:34.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:34.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:34.004 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:34.004 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:07:34.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:07:34.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:07:34.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:07:34.005 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:07:34.005 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:07:34.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:34.006 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:07:34.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:07:34.007 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:07:34.007 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:07:34.008 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:07:34.009 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:07:34.010 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:07:34.010 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:34.010 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.010 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:07:34.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:07:34.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:07:34.011 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ver-health 2026-02-13T07:07:34.012 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T07:07:34.012 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:34.012 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.012 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.164153 2026-02-13T07:07:34.013 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T07:07:34.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T07:07:34.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T07:07:34.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ver-health 1' TERM HUP INT 2026-02-13T07:07:34.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:178: TEST_check_version_health_3: run_mon td/ver-health a --public-addr=127.0.0.1:7165 2026-02-13T07:07:34.014 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ver-health 2026-02-13T07:07:34.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:07:34.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T07:07:34.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:07:34.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ver-health/a 2026-02-13T07:07:34.015 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ver-health/a --run-dir=td/ver-health --public-addr=127.0.0.1:7165 2026-02-13T07:07:34.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:07:34.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:34.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:34.034 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:34.034 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:34.034 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.034 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:34.035 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ver-health/a '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --mon-cluster-log-file=td/ver-health/log --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7165 2026-02-13T07:07:34.061 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:07:34.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T07:07:34.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:34.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:07:34.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:07:34.062 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:07:34.063 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:07:34.063 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:07:34.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:07:34.064 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:34.064 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:34.064 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.064 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.a.asok 2026-02-13T07:07:34.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:34.064 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.a.asok config get fsid 2026-02-13T07:07:34.107 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T07:07:34.107 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:34.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T07:07:34.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:07:34.109 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:07:34.109 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T07:07:34.109 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T07:07:34.109 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T07:07:34.109 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:34.109 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:34.109 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.110 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.a.asok 2026-02-13T07:07:34.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:34.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.a.asok config get mon_host 2026-02-13T07:07:34.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:179: TEST_check_version_health_3: run_mon td/ver-health b --public-addr=127.0.0.1:7166 2026-02-13T07:07:34.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ver-health 2026-02-13T07:07:34.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T07:07:34.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T07:07:34.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T07:07:34.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ver-health/b 2026-02-13T07:07:34.154 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ver-health/b --run-dir=td/ver-health --public-addr=127.0.0.1:7166 2026-02-13T07:07:34.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T07:07:34.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:34.174 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:34.174 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:34.174 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:34.174 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.175 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:34.175 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ver-health/b '--log-file=td/ver-health/$name.log' '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --mon-cluster-log-file=td/ver-health/log --run-dir=td/ver-health '--pid-file=td/ver-health/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7166 2026-02-13T07:07:34.199 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T07:07:34.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T07:07:34.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:34.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T07:07:34.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T07:07:34.200 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T07:07:34.202 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T07:07:34.202 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T07:07:34.202 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T07:07:34.202 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:34.202 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:34.202 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.202 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.b.asok 2026-02-13T07:07:34.202 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:34.202 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.b.asok config get fsid 2026-02-13T07:07:34.247 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T07:07:34.247 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T07:07:34.247 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T07:07:34.247 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T07:07:34.247 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T07:07:34.247 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T07:07:34.247 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T07:07:34.247 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T07:07:34.248 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T07:07:34.248 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:34.248 INFO:tasks.workunit.client.0.trial040.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.248 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.164153/ceph-mon.b.asok 2026-02-13T07:07:34.248 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T07:07:34.248 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.164153/ceph-mon.b.asok config get mon_host 2026-02-13T07:07:34.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:181: TEST_check_version_health_3: local start_osd_time=60 2026-02-13T07:07:34.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:183: TEST_check_version_health_3: EXTRA_OPTS=' --osd-objectstore=memstore' 2026-02-13T07:07:34.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:183: TEST_check_version_health_3: run_osd td/ver-health 0 2026-02-13T07:07:34.293 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ver-health 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ver-health/0 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ver-health/0' 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ver-health/0/journal' 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:07:34.294 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --osd-objectstore=memstore' 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:07:34.295 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:07:34.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:34.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:07:34.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:07:34.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:07:34.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:07:34.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:07:34.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:07:34.296 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ver-health/0 2026-02-13T07:07:34.296 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:07:34.297 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=79c88cbc-644b-4d14-be32-9771d4dcc74c 2026-02-13T07:07:34.297 INFO:tasks.workunit.client.0.trial040.stdout:add osd0 79c88cbc-644b-4d14-be32-9771d4dcc74c 2026-02-13T07:07:34.297 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 79c88cbc-644b-4d14-be32-9771d4dcc74c' 2026-02-13T07:07:34.298 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:07:34.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC2zY5pxaggEhAAy+eOF6T42Acge7gfvqRbYw== 2026-02-13T07:07:34.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC2zY5pxaggEhAAy+eOF6T42Acge7gfvqRbYw=="}' 2026-02-13T07:07:34.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 79c88cbc-644b-4d14-be32-9771d4dcc74c -i td/ver-health/0/new.json 2026-02-13T07:07:40.413 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:40.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ver-health/0/new.json 2026-02-13T07:07:40.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/0 --osd-journal=td/ver-health/0/journal --chdir= --osd-objectstore=memstore --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC2zY5pxaggEhAAy+eOF6T42Acge7gfvqRbYw== --osd-uuid 79c88cbc-644b-4d14-be32-9771d4dcc74c 2026-02-13T07:07:40.439 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:40.436+0000 7f70a68b4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:40.440 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:40.438+0000 7f70a68b4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:40.441 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:40.439+0000 7f70a68b4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:40.442 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:40.439+0000 7f70a68b4900 -1 memstore(td/ver-health/0) td/ver-health/0 2026-02-13T07:07:40.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ver-health/0/keyring 2026-02-13T07:07:40.447 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:07:40.448 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T07:07:40.448 INFO:tasks.workunit.client.0.trial040.stdout:adding osd0 key to auth repository 2026-02-13T07:07:40.448 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ver-health/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:07:40.550 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:07:40.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T07:07:40.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/0 --osd-journal=td/ver-health/0/journal --chdir= --osd-objectstore=memstore --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:07:40.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:07:40.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:07:40.551 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:07:40.568 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:40.565+0000 7f97696a8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:40.569 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:40.566+0000 7f97696a8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:40.570 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:40.567+0000 7f97696a8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:40.571 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:40.569+0000 7f97696a8900 -1 Falling back to public interface 2026-02-13T07:07:40.576 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:40.574+0000 7f97696a8900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:07:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T07:07:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:07:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:07:40.648 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:07:40.649 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:07:40.649 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:40.649 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:40.649 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:07:40.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:40.650 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:07:40.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:07:41.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:07:41.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:41.748 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:41.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:07:41.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:41.749 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:07:41.847 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/32450258,v1:127.0.0.1:6801/32450258] [v2:127.0.0.1:6802/32450258,v1:127.0.0.1:6803/32450258] exists,up 79c88cbc-644b-4d14-be32-9771d4dcc74c 2026-02-13T07:07:41.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:07:41.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:07:41.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:07:41.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:184: TEST_check_version_health_3: EXTRA_OPTS=' --osd-objectstore=memstore' 2026-02-13T07:07:41.851 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:184: TEST_check_version_health_3: run_osd td/ver-health 1 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ver-health 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ver-health/1 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ver-health/1' 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ver-health/1/journal' 2026-02-13T07:07:41.852 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --osd-objectstore=memstore' 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:07:41.853 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ver-health/1 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:07:41.854 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=be9b85b6-6eed-48be-acca-bf5d5037d2ec 2026-02-13T07:07:41.855 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 be9b85b6-6eed-48be-acca-bf5d5037d2ec' 2026-02-13T07:07:41.855 INFO:tasks.workunit.client.0.trial040.stdout:add osd1 be9b85b6-6eed-48be-acca-bf5d5037d2ec 2026-02-13T07:07:41.855 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:07:41.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC9zY5pZGdPMxAAMB4sglyw7Oey9oiBvA/X/w== 2026-02-13T07:07:41.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC9zY5pZGdPMxAAMB4sglyw7Oey9oiBvA/X/w=="}' 2026-02-13T07:07:41.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new be9b85b6-6eed-48be-acca-bf5d5037d2ec -i td/ver-health/1/new.json 2026-02-13T07:07:41.967 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:41.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ver-health/1/new.json 2026-02-13T07:07:41.975 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/1 --osd-journal=td/ver-health/1/journal --chdir= --osd-objectstore=memstore --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC9zY5pZGdPMxAAMB4sglyw7Oey9oiBvA/X/w== --osd-uuid be9b85b6-6eed-48be-acca-bf5d5037d2ec 2026-02-13T07:07:41.993 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:41.991+0000 7fb0d94ec900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:41.994 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:41.992+0000 7fb0d94ec900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:41.995 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:41.993+0000 7fb0d94ec900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:41.996 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:41.993+0000 7fb0d94ec900 -1 memstore(td/ver-health/1) td/ver-health/1 2026-02-13T07:07:42.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ver-health/1/keyring 2026-02-13T07:07:42.001 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:07:42.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T07:07:42.002 INFO:tasks.workunit.client.0.trial040.stdout:adding osd1 key to auth repository 2026-02-13T07:07:42.002 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ver-health/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:07:42.105 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:07:42.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T07:07:42.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/1 --osd-journal=td/ver-health/1/journal --chdir= --osd-objectstore=memstore --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:07:42.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:07:42.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:07:42.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:07:42.123 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:42.121+0000 7fdf9a072900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:42.125 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:42.122+0000 7fdf9a072900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:42.126 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:42.123+0000 7fdf9a072900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:42.127 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:42.124+0000 7fdf9a072900 -1 Falling back to public interface 2026-02-13T07:07:42.134 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:42.131+0000 7fdf9a072900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:07:42.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T07:07:42.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:07:42.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:07:42.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:07:42.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:07:42.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:42.206 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:07:42.207 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:42.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:42.207 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:07:42.306 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:07:43.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:07:43.307 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:43.308 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:43.308 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:07:43.309 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:43.309 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:07:43.405 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/998473295,v1:127.0.0.1:6809/998473295] [v2:127.0.0.1:6810/998473295,v1:127.0.0.1:6811/998473295] exists,up be9b85b6-6eed-48be-acca-bf5d5037d2ec 2026-02-13T07:07:43.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:07:43.409 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:185: TEST_check_version_health_3: EXTRA_OPTS=' --osd-objectstore=memstore' 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:185: TEST_check_version_health_3: run_osd td/ver-health 2 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ver-health 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ver-health/2 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:07:43.410 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ver-health/2' 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ver-health/2/journal' 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --osd-objectstore=memstore' 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:43.411 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T07:07:43.412 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T07:07:43.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ver-health/2 2026-02-13T07:07:43.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T07:07:43.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=96967d62-17d9-4b51-8a05-877256ef2c65 2026-02-13T07:07:43.413 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 96967d62-17d9-4b51-8a05-877256ef2c65' 2026-02-13T07:07:43.413 INFO:tasks.workunit.client.0.trial040.stdout:add osd2 96967d62-17d9-4b51-8a05-877256ef2c65 2026-02-13T07:07:43.413 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T07:07:43.420 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC/zY5pbpf1GBAAIgENiTHFUfmd6fmS73CVsQ== 2026-02-13T07:07:43.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC/zY5pbpf1GBAAIgENiTHFUfmd6fmS73CVsQ=="}' 2026-02-13T07:07:43.421 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 96967d62-17d9-4b51-8a05-877256ef2c65 -i td/ver-health/2/new.json 2026-02-13T07:07:43.524 INFO:tasks.workunit.client.0.trial040.stdout:2 2026-02-13T07:07:43.530 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ver-health/2/new.json 2026-02-13T07:07:43.531 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/2 --osd-journal=td/ver-health/2/journal --chdir= --osd-objectstore=memstore --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC/zY5pbpf1GBAAIgENiTHFUfmd6fmS73CVsQ== --osd-uuid 96967d62-17d9-4b51-8a05-877256ef2c65 2026-02-13T07:07:43.550 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:43.548+0000 7f6560be0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:43.552 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:43.549+0000 7f6560be0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:43.553 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:43.550+0000 7f6560be0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:43.553 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:43.551+0000 7f6560be0900 -1 memstore(td/ver-health/2) td/ver-health/2 2026-02-13T07:07:43.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ver-health/2/keyring 2026-02-13T07:07:43.558 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T07:07:43.559 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T07:07:43.559 INFO:tasks.workunit.client.0.trial040.stdout:adding osd2 key to auth repository 2026-02-13T07:07:43.560 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ver-health/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T07:07:43.664 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:07:43.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T07:07:43.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/2 --osd-journal=td/ver-health/2/journal --chdir= --osd-objectstore=memstore --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:07:43.665 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T07:07:43.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T07:07:43.666 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T07:07:43.682 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:43.679+0000 7f45fe00f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:43.684 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:43.681+0000 7f45fe00f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:43.684 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:43.682+0000 7f45fe00f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:43.686 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:43.683+0000 7f45fe00f900 -1 Falling back to public interface 2026-02-13T07:07:43.693 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:43.690+0000 7f45fe00f900 -1 osd.2 0 log_to_monitors true 2026-02-13T07:07:43.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T07:07:43.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:07:43.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:07:43.763 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:07:43.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:07:43.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:43.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:07:43.764 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:43.764 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:43.765 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:07:43.862 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:07:44.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:07:44.863 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:44.864 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:07:44.864 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:07:44.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:44.865 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:07:44.962 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/817980846,v1:127.0.0.1:6817/817980846] [v2:127.0.0.1:6818/817980846,v1:127.0.0.1:6819/817980846] exists,up 96967d62-17d9-4b51-8a05-877256ef2c65 2026-02-13T07:07:44.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:07:44.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:07:44.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:07:44.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:187: TEST_check_version_health_3: local warn_older_version_delay=30 2026-02-13T07:07:44.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:189: TEST_check_version_health_3: sleep 5 2026-02-13T07:07:49.963 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:190: TEST_check_version_health_3: ceph health detail 2026-02-13T07:07:50.060 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_OK 2026-02-13T07:07:50.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:192: TEST_check_version_health_3: ceph health detail 2026-02-13T07:07:50.067 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:192: TEST_check_version_health_3: grep DAEMON_OLD_VERSION 2026-02-13T07:07:50.169 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:193: TEST_check_version_health_3: ceph tell 'mon.*' injectargs '--mon_warn_older_version_delay 30' 2026-02-13T07:07:50.221 INFO:tasks.workunit.client.0.trial040.stdout:mon.a: {} 2026-02-13T07:07:50.229 INFO:tasks.workunit.client.0.trial040.stdout:mon.b: {} 2026-02-13T07:07:50.233 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:194: TEST_check_version_health_3: kill_daemons td/ver-health KILL osd.1 2026-02-13T07:07:50.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:07:50.234 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:07:50.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:07:50.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:07:50.234 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:07:50.342 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:07:50.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:195: TEST_check_version_health_3: EXTRA_OPTS=' --osd-objectstore=memstore' 2026-02-13T07:07:50.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:195: TEST_check_version_health_3: ceph_debug_version_for_testing=01.00.00-gversion-test 2026-02-13T07:07:50.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:195: TEST_check_version_health_3: activate_osd td/ver-health 1 2026-02-13T07:07:50.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ver-health 2026-02-13T07:07:50.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:07:50.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T07:07:50.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:07:50.343 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ver-health/1 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ver-health/1' 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ver-health/1/journal' 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+=' --osd-objectstore=memstore' 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:07:50.344 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:07:50.345 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:07:50.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:07:50.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:07:50.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:07:50.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:07:50.346 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ver-health/1 2026-02-13T07:07:50.346 INFO:tasks.workunit.client.0.trial040.stdout:start osd.1 2026-02-13T07:07:50.347 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T07:07:50.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/1 --osd-journal=td/ver-health/1/journal --chdir= --osd-objectstore=memstore --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:07:50.348 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ver-health/1/whoami 2026-02-13T07:07:50.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T07:07:50.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:07:50.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:07:50.348 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:07:50.359 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:50.356+0000 7fab8d73d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:50.361 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:50.358+0000 7fab8d73d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:50.361 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:50.359+0000 7fab8d73d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:07:50.363 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:50.360+0000 7fab8d73d900 -1 Falling back to public interface 2026-02-13T07:07:50.370 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:07:50.367+0000 7fab8d73d900 -1 osd.1 0 log_to_monitors true 2026-02-13T07:07:50.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T07:07:50.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:07:50.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T07:07:50.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:07:50.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:07:50.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:07:50.444 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:07:50.445 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:07:50.445 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:07:50.445 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T07:07:50.542 INFO:tasks.workunit.client.0.trial040.stdout:osd.1 up in weight 1 up_from 19 up_thru 0 down_at 17 last_clean_interval [0,0) [v2:127.0.0.1:6808/4267970092,v1:127.0.0.1:6809/4267970092] [v2:127.0.0.1:6810/4267970092,v1:127.0.0.1:6811/4267970092] exists,up be9b85b6-6eed-48be-acca-bf5d5037d2ec 2026-02-13T07:07:50.542 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:07:50.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:07:50.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:07:50.543 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:200: TEST_check_version_health_3: sleep 10 2026-02-13T07:08:00.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:202: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:00.544 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:202: TEST_check_version_health_3: grep DAEMON_OLD_VERSION 2026-02-13T07:08:00.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:205: TEST_check_version_health_3: wait_for_health_string 'HEALTH_WARN .*There is a daemon running an older version of ceph' 20 2026-02-13T07:08:00.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:40: wait_for_health_string: local 'grep_string=HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:00.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:41: wait_for_health_string: local seconds=20 2026-02-13T07:08:00.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:44: wait_for_health_string: set -o pipefail 2026-02-13T07:08:00.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:45: wait_for_health_string: PASSED=false 2026-02-13T07:08:00.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i=0 )) 2026-02-13T07:08:00.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:00.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:00.644 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:00.745 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:01.746 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:01.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:01.747 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:01.748 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:01.848 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:02.849 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:02.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:02.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:02.850 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:02.950 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:03.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:03.951 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:03.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:03.952 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:04.056 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:05.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:05.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:05.057 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:05.058 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:05.156 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:06.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:06.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:06.157 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:06.158 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:06.257 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:07.258 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:07.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:07.259 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:07.260 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:07.359 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:08.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:08.360 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:08.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:08.361 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:08.461 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:09.462 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:09.462 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:09.463 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:09.463 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:09.565 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:10.566 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:10.567 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:10.567 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:10.567 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:10.668 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:11.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:11.669 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:11.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:11.670 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:11.768 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:12.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:12.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:12.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:12.770 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:12.871 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:13.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:13.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:13.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:13.873 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:13.972 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:14.973 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:14.973 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:14.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:14.974 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:15.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:49: wait_for_health_string: PASSED=true 2026-02-13T07:08:15.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:50: wait_for_health_string: break 2026-02-13T07:08:15.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:54: wait_for_health_string: set +o pipefail 2026-02-13T07:08:15.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:57: wait_for_health_string: '[' true = false ']' 2026-02-13T07:08:15.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:61: wait_for_health_string: return 0 2026-02-13T07:08:15.073 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:207: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:15.168 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_WARN There is a daemon running an older version of ceph 2026-02-13T07:08:15.168 INFO:tasks.workunit.client.0.trial040.stdout:[WRN] DAEMON_OLD_VERSION: There is a daemon running an older version of ceph 2026-02-13T07:08:15.168 INFO:tasks.workunit.client.0.trial040.stdout: osd.1 is running an older version of ceph: 01.00.00-gversion-test 2026-02-13T07:08:15.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:209: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:15.174 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:209: TEST_check_version_health_3: grep -q 'HEALTH_WARN .*There is a daemon running an older version of ceph' 2026-02-13T07:08:15.274 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:210: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:15.274 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:210: TEST_check_version_health_3: grep -q '^[[]WRN[]] DAEMON_OLD_VERSION: There is a daemon running an older version of ceph' 2026-02-13T07:08:15.375 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:211: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:15.375 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:211: TEST_check_version_health_3: grep -q 'osd.1 is running an older version of ceph: 01.00.00-gversion-test' 2026-02-13T07:08:15.473 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:213: TEST_check_version_health_3: kill_daemons td/ver-health KILL osd.2 2026-02-13T07:08:15.474 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:08:15.474 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:08:15.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:08:15.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:08:15.474 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:214: TEST_check_version_health_3: ceph_debug_version_for_testing=01.00.00-gversion-test 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:214: TEST_check_version_health_3: activate_osd td/ver-health 2 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ver-health 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ver-health/2 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:08:15.583 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ver-health/2' 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ver-health/2/journal' 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:08:15.584 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:08:15.585 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ver-health/2 2026-02-13T07:08:15.586 INFO:tasks.workunit.client.0.trial040.stdout:start osd.2 2026-02-13T07:08:15.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T07:08:15.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/2 --osd-journal=td/ver-health/2/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:08:15.587 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ver-health/2/whoami 2026-02-13T07:08:15.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T07:08:15.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:08:15.587 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:08:15.588 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:08:15.600 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:15.599+0000 7fe2d1822900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:08:15.602 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:15.600+0000 7fe2d1822900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:08:15.603 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:15.601+0000 7fe2d1822900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:08:15.604 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:15.602+0000 7fe2d1822900 -1 Falling back to public interface 2026-02-13T07:08:15.612 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:15.610+0000 7fe2d1822900 -1 osd.2 0 log_to_monitors true 2026-02-13T07:08:15.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T07:08:15.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:08:15.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T07:08:15.684 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:08:15.684 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:08:15.684 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:08:15.684 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:08:15.684 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:08:15.685 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:08:15.685 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:08:15.779 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:08:16.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:08:16.780 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:08:16.781 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:08:16.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:08:16.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:08:16.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T07:08:16.877 INFO:tasks.workunit.client.0.trial040.stdout:osd.2 up in weight 1 up_from 22 up_thru 0 down_at 20 last_clean_interval [0,0) [v2:127.0.0.1:6816/1958251814,v1:127.0.0.1:6817/1958251814] [v2:127.0.0.1:6818/1958251814,v1:127.0.0.1:6819/1958251814] exists,up 96967d62-17d9-4b51-8a05-877256ef2c65 2026-02-13T07:08:16.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:08:16.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:08:16.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:08:16.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:215: TEST_check_version_health_3: kill_daemons td/ver-health KILL osd.0 2026-02-13T07:08:16.879 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:08:16.879 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:08:16.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:08:16.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:08:16.879 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:08:16.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:08:16.988 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:216: TEST_check_version_health_3: ceph_debug_version_for_testing=02.00.00-gversion-test 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:216: TEST_check_version_health_3: activate_osd td/ver-health 0 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ver-health 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ver-health/0 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 ' 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T07:08:16.989 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ver-health/0' 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ver-health/0/journal' 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ver-health' 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:08:16.990 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ver-health/$name.log' 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ver-health/$name.pid' 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T07:08:16.991 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ver-health/0 2026-02-13T07:08:16.992 INFO:tasks.workunit.client.0.trial040.stdout:start osd.0 2026-02-13T07:08:16.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T07:08:16.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1a08d5b0-4b04-4caf-b30d-8617bf07f864 --auth-supported=none --mon-host= --mon_health_to_clog_tick_interval=1.0 --mon-host=127.0.0.1:7165 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ver-health/0 --osd-journal=td/ver-health/0/journal --chdir= --run-dir=td/ver-health '--admin-socket=/tmp/ceph-asok.164153/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ver-health/$name.log' '--pid-file=td/ver-health/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T07:08:16.993 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ver-health/0/whoami 2026-02-13T07:08:16.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T07:08:16.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T07:08:16.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T07:08:16.993 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T07:08:17.006 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:17.004+0000 7f8200c6b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:08:17.008 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:17.006+0000 7f8200c6b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:08:17.009 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:17.007+0000 7f8200c6b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T07:08:17.010 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:17.008+0000 7f8200c6b900 -1 Falling back to public interface 2026-02-13T07:08:17.015 INFO:tasks.workunit.client.0.trial040.stderr:2026-02-13T07:08:17.014+0000 7f8200c6b900 -1 osd.0 0 log_to_monitors true 2026-02-13T07:08:17.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T07:08:17.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T07:08:17.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T07:08:17.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T07:08:17.088 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T07:08:17.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:08:17.089 INFO:tasks.workunit.client.0.trial040.stdout:0 2026-02-13T07:08:17.089 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T07:08:17.090 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:08:17.090 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:08:17.184 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T07:08:18.185 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T07:08:18.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T07:08:18.186 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T07:08:18.186 INFO:tasks.workunit.client.0.trial040.stdout:1 2026-02-13T07:08:18.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T07:08:18.187 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T07:08:18.285 INFO:tasks.workunit.client.0.trial040.stdout:osd.0 up in weight 1 up_from 25 up_thru 0 down_at 23 last_clean_interval [0,0) [v2:127.0.0.1:6800/844476677,v1:127.0.0.1:6801/844476677] [v2:127.0.0.1:6802/844476677,v1:127.0.0.1:6803/844476677] exists,up 79c88cbc-644b-4d14-be32-9771d4dcc74c 2026-02-13T07:08:18.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T07:08:18.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T07:08:18.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T07:08:18.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:218: TEST_check_version_health_3: wait_for_health_string 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:08:18.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:40: wait_for_health_string: local 'grep_string=HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:08:18.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:41: wait_for_health_string: local seconds=20 2026-02-13T07:08:18.287 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:44: wait_for_health_string: set -o pipefail 2026-02-13T07:08:18.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:45: wait_for_health_string: PASSED=false 2026-02-13T07:08:18.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i=0 )) 2026-02-13T07:08:18.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:18.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:18.288 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:08:18.384 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:52: wait_for_health_string: sleep 1 2026-02-13T07:08:19.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i++ )) 2026-02-13T07:08:19.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:46: wait_for_health_string: (( i < 20 )) 2026-02-13T07:08:19.385 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: ceph health 2026-02-13T07:08:19.386 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:47: wait_for_health_string: grep -q 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:08:19.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:49: wait_for_health_string: PASSED=true 2026-02-13T07:08:19.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:50: wait_for_health_string: break 2026-02-13T07:08:19.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:54: wait_for_health_string: set +o pipefail 2026-02-13T07:08:19.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:57: wait_for_health_string: '[' true = false ']' 2026-02-13T07:08:19.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:61: wait_for_health_string: return 0 2026-02-13T07:08:19.483 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:220: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:19.576 INFO:tasks.workunit.client.0.trial040.stdout:HEALTH_ERR There are daemons running multiple old versions of ceph 2026-02-13T07:08:19.576 INFO:tasks.workunit.client.0.trial040.stdout:[ERR] DAEMON_OLD_VERSION: There are daemons running multiple old versions of ceph 2026-02-13T07:08:19.576 INFO:tasks.workunit.client.0.trial040.stdout: osd.1 osd.2 are running an older version of ceph: 01.00.00-gversion-test 2026-02-13T07:08:19.576 INFO:tasks.workunit.client.0.trial040.stdout: osd.0 is running an older version of ceph: 02.00.00-gversion-test 2026-02-13T07:08:19.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:221: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:19.582 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:221: TEST_check_version_health_3: grep -q 'HEALTH_ERR .*There are daemons running multiple old versions of ceph' 2026-02-13T07:08:19.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:222: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:19.683 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:222: TEST_check_version_health_3: grep -q '^[[]ERR[]] DAEMON_OLD_VERSION: There are daemons running multiple old versions of ceph' 2026-02-13T07:08:19.781 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:223: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:19.782 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:223: TEST_check_version_health_3: grep -q 'osd.1 osd.2 are running an older version of ceph: 01.00.00-gversion-test' 2026-02-13T07:08:19.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:224: TEST_check_version_health_3: ceph health detail 2026-02-13T07:08:19.882 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:224: TEST_check_version_health_3: grep -q 'osd.0 is running an older version of ceph: 02.00.00-gversion-test' 2026-02-13T07:08:19.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ver-health.sh:35: run: teardown td/ver-health 2026-02-13T07:08:19.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:08:19.984 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T07:08:19.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:08:19.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:08:19.985 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:08:19.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:08:19.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:08:19.985 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:08:20.091 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:08:20.092 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:08:20.092 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:08:20.093 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:08:20.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:08:20.094 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:08:20.095 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:08:20.096 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:08:20.096 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:08:20.096 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:08:20.096 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:08:20.097 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:08:20.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T07:08:20.098 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:08:20.103 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:08:20.103 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:08:20.103 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:08:20.103 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:08:20.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:08:20.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:08:20.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-13T07:08:20.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/ver-health 0 2026-02-13T07:08:20.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ver-health 2026-02-13T07:08:20.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T07:08:20.105 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ver-health KILL 2026-02-13T07:08:20.105 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T07:08:20.105 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T07:08:20.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T07:08:20.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T07:08:20.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T07:08:20.106 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T07:08:20.107 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T07:08:20.107 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T07:08:20.108 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T07:08:20.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T07:08:20.108 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T07:08:20.109 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T07:08:20.109 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:08:20.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T07:08:20.110 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T07:08:20.110 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T07:08:20.111 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:08:20.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T07:08:20.112 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ver-health 2026-02-13T07:08:20.113 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T07:08:20.113 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T07:08:20.113 INFO:tasks.workunit.client.0.trial040.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.164153 2026-02-13T07:08:20.113 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.164153 2026-02-13T07:08:20.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T07:08:20.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T07:08:20.114 INFO:tasks.workunit.client.0.trial040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-13T07:08:20.115 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T07:08:20.115 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T07:08:20.140 INFO:tasks.workunit:Stopping ['misc'] on client.0... 2026-02-13T07:08:20.140 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-13T07:08:20.497 DEBUG:teuthology.parallel:result is None 2026-02-13T07:08:20.497 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-13T07:08:20.521 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-13T07:08:20.521 DEBUG:teuthology.orchestra.run.trial040:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-13T07:08:20.577 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-13T07:08:20.577 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-13T07:08:20.593 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-13T07:08:20.593 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-13T07:08:20.745 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-13T07:08:20.757 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-13T07:08:20.757 DEBUG:teuthology.orchestra.run.trial040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T07:08:20.774 INFO:teuthology.orchestra.run.trial040.stderr:bash: line 1: ntpq: command not found 2026-02-13T07:08:20.805 INFO:teuthology.orchestra.run.trial040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T07:08:20.806 INFO:teuthology.orchestra.run.trial040.stdout:=============================================================================== 2026-02-13T07:08:20.806 INFO:teuthology.orchestra.run.trial040.stdout:^+ connected.by.freedominte> 2 7 377 57 -560us[ -560us] +/- 41ms 2026-02-13T07:08:20.806 INFO:teuthology.orchestra.run.trial040.stdout:^+ time2.trtnw.net 2 6 377 56 -1033us[-1033us] +/- 38ms 2026-02-13T07:08:20.806 INFO:teuthology.orchestra.run.trial040.stdout:^+ kjsl-fmt2-net.fmt2.kjsl.> 2 6 377 58 +729us[ +729us] +/- 33ms 2026-02-13T07:08:20.806 INFO:teuthology.orchestra.run.trial040.stdout:^* h134-215-155-177.mdtnwi.> 2 7 377 60 +270us[ +293us] +/- 23ms 2026-02-13T07:08:20.807 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-13T07:08:20.818 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-13T07:08:20.819 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-13T07:08:20.840 DEBUG:teuthology.orchestra.run.trial040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-13T07:08:20.888 DEBUG:teuthology.orchestra.run.trial040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-13T07:08:20.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T07:08:20.955 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-13T07:08:20.970 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-13T07:08:20.985 INFO:teuthology.task.internal:Duration was 2011.151400 seconds 2026-02-13T07:08:20.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-13T07:08:21.008 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-13T07:08:21.008 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-13T07:08:21.044 INFO:teuthology.orchestra.run.trial040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T07:08:21.220 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-13T07:08:21.220 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial040.front.sepia.ceph.com 2026-02-13T07:08:21.221 DEBUG:teuthology.orchestra.run.trial040:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-13T07:08:21.279 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-13T07:08:21.279 DEBUG:teuthology.orchestra.run.trial040:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T07:08:21.360 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-13T07:08:21.360 DEBUG:teuthology.orchestra.run.trial040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T07:08:21.422 INFO:teuthology.orchestra.run.trial040.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T07:08:21.424 INFO:teuthology.orchestra.run.trial040.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T07:08:21.424 INFO:teuthology.orchestra.run.trial040.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-13T07:08:21.424 INFO:teuthology.orchestra.run.trial040.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T07:08:21.424 INFO:teuthology.orchestra.run.trial040.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-13T07:08:21.427 INFO:teuthology.orchestra.run.trial040.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 87.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-13T07:08:21.429 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-13T07:08:21.445 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-13T07:08:21.445 DEBUG:teuthology.orchestra.run.trial040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-13T07:08:21.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-13T07:08:21.508 DEBUG:teuthology.orchestra.run.trial040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:08:21.555 INFO:teuthology.orchestra.run.trial040.stdout:kernel.core_pattern = core 2026-02-13T07:08:21.566 DEBUG:teuthology.orchestra.run.trial040:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-13T07:08:21.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T07:08:21.621 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-13T07:08:21.638 INFO:teuthology.task.internal:Transferring archived files... 2026-02-13T07:08:21.640 DEBUG:teuthology.misc:Transferring archived files from trial040:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-13_01:43:11-rados-wip-bharath17-testing-2026-02-12-0618-tentacle-distro-default-trial/47825/remote/trial040 2026-02-13T07:08:21.642 DEBUG:teuthology.orchestra.run.trial040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-13T07:08:21.695 INFO:teuthology.task.internal:Removing archive directory... 2026-02-13T07:08:21.695 DEBUG:teuthology.orchestra.run.trial040:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-13T07:08:21.742 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-13T07:08:21.755 INFO:teuthology.task.internal:Not uploading archives. 2026-02-13T07:08:21.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-13T07:08:21.770 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-13T07:08:21.770 DEBUG:teuthology.orchestra.run.trial040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-13T07:08:21.797 INFO:teuthology.orchestra.run.trial040.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 07:08 /home/ubuntu/cephtest 2026-02-13T07:08:21.798 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-13T07:08:21.809 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-13T07:08:21.839 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/misc} duration: 2011.1513996124268 flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com success: true 2026-02-13T07:08:21.839 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T07:08:21.917 INFO:teuthology.run:pass